3D Model Construction System for Personalized Rehabilitation Prosthetics Based on Machine Vision

: At present, the molding process of domestic prosthetic sockets is still mainly made by hand. According to statistics, the number of various types of disabilities in my country has reached 85.02 million, of which more than 6 million are disabled in the lower limbs. Therefore, the installation of prosthetics has become an important means of compensating for missing limbs. , but the old process has many problems and is complicated and backward, heavy, poor performance, long production process cycle and not environmentally friendly, and the appearance is not beautiful. These problems can no longer meet the needs of patients in the current society. At the same time, during the "13th Five-Year Plan" period, rehabilitation medicine has become a key industry promoted by the country, and the rehabilitation appliance industry has also developed rapidly. With the promotion of policies and technologies, the market value and space of rehabilitation appliances have shown a good form. Therefore, the research of this project In line with the development trend of the times.This work proposes a personalized rehabilitation prosthetic 3D model construction system based on machine vision. In the era of artificial intelligence, the XTOM 3D scanner under machine vision scans the surface contour of the patient's stump to obtain its point cloud information. The first starting point is innovation, providing a data basis for socket design to improve scanning accuracy and find the most suitable socket structure. Finally, according to the individual needs of the user, the neural network is used to extract video features, analyze the most reasonable movement mode of the patient, and achieve the research on the combination of user behavior and product design, so as to analyze the force of the socket in the later stage. The part of the force analysis is innovated again. By adjusting the gravity distribution of the prosthesis, the angle of force, and the angle of the metal connector, the prosthesis printed by the 3D model for the subsequent industrial-grade 3D printer meets the patient's requirements.


Research Background
In recent years, due to various diseases, wars, accidents and other disasters, tens of millions of amputees have been brought to the society, among which the lower limb disabilities accounted for about 70% [1]. Therefore, prosthetics play an important role as a replacement for missing limbs for people with residual limbs, so that they can recover to a certain degree of self-care ability. On the other hand, the number of employees in my country's prosthetic orthotics industry is less than 5,000, of which there are less than 1,000 professional and technical personnel. The traditional orthotics manufacturing technology is highly specialized, involves a wide range of knowledge, and requires high technical personnel. Prosthetics and orthotics There is a serious shortage of assembly personnel. Most of the front-line technicians are trained by masters and apprentices. Moreover, technicians and technicians at the university and technical secondary school levels have narrow professional scope and poor foundation. needs. The emergence of 3D printing technology has lowered the technical threshold for the production of orthotics, allowing ordinary doctors to master the method of 3D printing to make rehabilitation orthotics after simple training, which can also alleviate the current shortage of personnel to a certain extent. With the rapid development of machine vision, 3D reconstruction can restore the 3D structure by calculating the 2D image. Compared with the 2D image method, the 3D structure is more intuitive and three-dimensional, and it provides certain technical support for personalized rehabilitation design. Therefore, this study focuses on the method of 3D printing design and production, and develops a 3D printing design and production system suitable for popularization.

Research status at home and abroad
3D printing technology is known as the prelude to the industrial revolution 4.0, and it has been widely used in medical treatment abroad. Anthony Atala of Wake Forest University in the United States has realized 3D printing of kidneys through 3D technology, and Cornell University has developed a 3D printing artificial ear [2]. Wang [3] et al. studied the application of 3D printing to complex tissue reconstruction. With regard to personalized customization, Shih [4] proposed remote design and manufacture of orthotics, which improved the manufacturing efficiency of personalized orthotics.
For China, 3D printing technology is introduced late, and there are few 3D printing medical auxiliary equipment for prosthetic sockets. Cheng Siyuan [5] et al. used 3D scanning as the method of data acquisition of the injured part, combined with the personalized design method, designed and manufactured a personalized arm splint model. Chen Ruiying [6] designed a barefoot orthosis using a parametric design method.
For the research on 3D reconstruction under machine vision, foreign countries started earlier. L.G Roberts explored the restoration of 2D images to 3D models through machine vision. Tomasi and others built the first complete 3D reconstruction and restoration system based on machine vision, which established the direction for the future development of 3D reconstruction. Although domestic research on 3D reconstruction algorithms started relatively late, it is developing rapidly. Harbin Institute of Technology has conducted in-depth research on the application of binocular stereo vision and realized intelligent autonomous navigation [7]. The National University of Defense Technology proposed a 3D reconstruction algorithm based on mapping depth solution and simplified iteration, and applied it to the 3D detection of vehicles [8]. With the continuous accumulation, our country will definitely make great progress in 3D reconstruction, which can just give a new opportunity for the development of 3D molding technology, making it have the characteristics of individualization and rapid prototyping. Work Difficulties and Innovation

Work Difficulty
3D printing, also known as additive manufacturing and additive manufacturing technology, is to input the model files designed by computer-aided design software [9] into the 3D printer, select different printing materials, and process them through fusion molding, laser sintering, melting, Stereoscopic stereolithography and other methods are used to print out the solid model, thereby completing the physical construction of the model, abandoning the production line, and not requiring traditional tools, fixtures, machine tools or any molds, thereby greatly reducing the cost and difficulty of manufacturing and processing. At present, plaster orthotics and plastic orthotics are mainly used. The production of traditional orthotics requires a lot of equipment and is highly dependent on the experience and skills of the production technicians. If the technicians are not experienced enough, the quality of the orthotics manufactured will be poor and the fitting rate will be low. It cannot match the shape of the patient well, and wearing inappropriate orthotics for a long time can easily induce complications such as skin ulcers.
Everyone's body structure is different, and the mechanism section of the patient who needs prosthetics is different. When using the prosthetic, due to the different walking posture, force, and angle, the force received inside the prosthetic is different, which is a test of the cross-section of the prosthetic. In order to improve the accuracy of strength, 3D scanning technology under machine vision and neural network feature extraction technology should be used reasonably to improve the accuracy of patients' residual limbs, and dynamic analysis and data redundancy processing of individual patient movement patterns can be performed to improve the accuracy of the system model. Personalized rehabilitation has greater practical significance.

Innovativeness of the work
With the development of society, science and technology, and materials, the hundreds of billions market of the rehabilitation appliance industry has been launched. The new models and new technologies of the rehabilitation appliance industry are constantly innovating. The market value and market space of the industry are also increasing, and the market prospect is showing a good situation. At present, most of the prosthetic limbs in my country are made by traditional technology, which is complicated and backward, with heavy weight, poor performance, unsightly appearance, low degree of standardization, non-environmental-friendly manufacturing process, and long production period, which is far behind the world level. Therefore, the development the production process and products of modern prosthetics have become an important work at present. The point cloud information obtained by the 3D scanner is the surface contour of the stump of the lower limbs of the human body, which provides basic data for the design of the socket, and the bone structure data related to the amputation site is required to design a suitable socket. According to the <Chinese Adult Human Dimensions>Standard and anatomical common data are used for data approximate fitting to obtain the digital information of the patient's residual limb. It is necessary to study the fitting ratio between the theoretical data and the scanning data, so as to serve as the basis for the selection of the prosthetic template model and the model configuration modification. Realize the purpose of personal customization.

3D Scanner Selection
For the 3D scanning part of this scheme design, the XTOM four-eye industrial-grade blue-ray high-precision 3D scanner of the Sanjiang Institute of Artificial Intelligence and Robotics of Yibin University was chosen for reverse modeling, as shown in Figure 1. The four-eye industrial-grade 3D scanner adopts advanced and stable system measuring heads, pioneering measurement technologies and algorithms, and provides accurate 3D measurement data to optimize the design process and improve industrial production processes.

3D Printer Selection
The application of the 3D printer in this design scheme is also an industrial-grade 3D printer of the type Xingkang SLA800 from the Sanjiang Institute of Artificial Intelligence and Robotics of Yibin University, as shown in Figure 2. SLA3D printing technology improves work efficiency. SLA focuses ultraviolet light of specific wavelength and intensity onto the surface of photocurable materials, making them solidify in sequence from point to line and from line to surface, thus completing the drawing of a layer section.

Work process
The process of this experiment is divided into the steps shown in Figure 3. Firstly, the collected video gait sequence is preprocessed, and secondly, the CNN network model and the LSTM network model are respectively constructed. First, CNN is used for spatial feature fusion extraction, and then LSTM performs time information feature extraction and then outputs.

. Convolutional Neural Network Structure Design
The residual network ResNet (Residual Networks) was proposed by Kaiming He et al. in 2015. It has a special network structure. He can build a deep neural network, but there will be no problems of gradient explosion and gradient disappearance. It works by passing the original input x directly to the output, and making it the input of the next layer. The final output of the network layer is F(x)+x. The realization of F(x)+x can be realized by a feed-forward neural network with "shortcut connections". Shortcut connections are connections that skip one or more layers. The core module of the ResNet model is called a ResNet block, the most common form of which consists of two sets of convolutions. Assuming that the input of the block is x and the output is z, and the calculation composed of convolutional BN and ReLU is a function F, then the calculation formula of the ResNet block is: = ( ( ( ) + ) In this study, a 34-layer convolution-pooling structure is adopted, and the size of the convolution kernel is small, which is convenient for parameter calculation, enough to analyze gait features, and can also ensure the robustness and accuracy of features. Set the network structure vertically from 2-layer to 3-layer LSTM network structure, which can ensure the effective analysis and identification of the temporal relationship of the video sequence. By analyzing the length of the gait video sequence, set the number of horizontal LSTM modules to 100. In this way, the timing information can be effectively fused, and after passing through the timing relationship of each layer, a multi-classification output is finally performed by using the SoftMax function.

Principle Analysis of Prosthetic Socket
Simplified mathematical model modeling in the socket: (1) The internal bone of the residual limb is simplified to a cylindrical structure; (2) The soft tissue is tapered under the socket pressure; (3) Soft tissue is regarded as a homogeneous and isotropic linear elastic material. The simplified model of the socket system is shown in Figure 6. The soft tissue is set as a linear elastic material, the elastic constant is K, and the compression amount is d. In the finite element analysis, the receiving cavity is set to be isotropic, and the elastic force of the interface micro-element is as follows:

=
The force of the simplified model is shown in Figure 7, which can be decomposed into elastic constants in the vertical and tangential directions, expressed by the following formula: Among them, E is the soft vertical elastic modulus, G is the shear modulus, t represents the uniform thickness of soft tissue, and A is the contact support area. Among them, the parameters involved in Equation 5.2 are calculated as follows: Where v is Poisson's ratio, 1 and 2 are the diameters of the upper and lower sides of the socket respectively, and L is the interface contact length. According to the theoretical analysis of the force of the prosthetic socket above, combined with the finite element analysis, the research on the forming process of the socket is achieved.

Establishment of double-layer hollow socket model based
The 3D model is obtained through reverse reconstruction, that is, after 3D scanning or CT/MRI scanning of the residual limb, the data is imported into SolidWorks, and the preliminary model is obtained through reverse reconstruction. The obtained preliminary residual limb socket model is still a hollow shell, and the result of meshing it can only generate a surface mesh, so it needs to be solidified, and solidification requires two steps: (1) Through SolidWorks software, the three-dimensional figure of the shell surface is generated into a solid; (2) Model the double-layer hollow structure of the prosthetic socket. In order to establish an accurate 3D printing geometric 3D model, the socket model is divided into finite element solid meshes, and the socket model after solid meshing is shown in Figure 10.

Finite Element Analysis of Prosthetic Socket
By setting the finite element model of the socket, dividing the mesh, defining the load and boundary conditions, and comparing the simulation results by Mises stress and Tresca stress.
As the equivalent stress of normal stress and shear stress, Mises can characterize the deformation yield strength of the 3D printed socket material, so as to guide the correction and adjustment. According to the distribution of force, the maximum Mises stress in the three stages is concentrated at the end of the prosthetic socket. In the stage of heel strike, the second largest pressure is located at the rear edge of the distal end of the socket; in the middle stage of stance, the force on the front and rear sides is equal; in the stage of toe off the ground, the second largest pressure is also located at the front edge of the socket.  Figure 12 is a typical stage displacement distribution map under Mises, that is, the moving distance of each part of the socket relative to the initial position. It can be seen that the maximum displacement occurs at the outer edge of the distal end of the socket, that is, the deformation is the largest. The simulation results conform to the biomechanical distribution characteristics of the thigh prosthesis.
Therefore, according to the force distribution of the Mises stress and displacement finite element simulation analysis, that is, the end of the 3D socket and the edge of the caliber are prone to rupture, and the above-mentioned areas can be modified. At the same time, appropriate smoothing corrections are made to the socket aperture ring to reduce edge stress concentration.
Compared with Mises, Tresca does not reflect the influence of intermediate principal stress on deformation, but it is also widely used in the analysis of force deformation [10]. According to the finite element force analysis, the maximum force is concentrated at the end of the socket, and the second force is at the rear side of the distal end, that is, the distribution trend of stress intensity is basically consistent with Mises, which is consistent with the biomechanical distribution of the thigh stump. Tresca is shown as the maximum shear force in the strength theory, and the shear force is represented by the maximum shear force in Abaqus/CAE.2018 12 、 13 、 21 、 23 、 31 、 32 ，( 12 = 21 ， 13 = 31 ， 23 = 32 ).1, 2, and 3 correspond to the x, y, and z axes, respectively). Wherein, the positive direction of the z-axis is from the far end to the end of the receiving cavity, the positive direction of the x-axis is the front side of the receiving cavity, and the positive direction of the y-axis is the right side.
This simulation selects S_12 shear force (the shear stress along the y-axis direction on the yz plane perpendicular to the x-axis) and S3 shear force (the shear stress along the z-axis direction on the yz plane) for research, and the finite element simulation results as shown in Figure 14.
It can be seen that the shear stress is mainly concentrated on the side wall of the socket. This is because the loaded socket creates a constant shear stress during the gait cycle. This force exists in the form of friction, mainly to resist gravity, so as to ensure that there will be no relative sliding between the contact interfaces of the socket when worn. Otherwise, the material will be destroyed when the shear strength is insufficient, which is very important in the production of socket molding. The finite element simulation results conform to the actual mechanical distribution.
It can be observed from the cloud images of the finite element stress distribution of the two deformation strength theories that the yield deformation distribution of the 3D socket is generally consistent and has a certain degree of accuracy. Stress concentration is prone to occur at the end of the 3D socket and the edge area of the caliber. In these areas with large yield, the 3D socket is prone to cracking and damage. Appropriate modification can be performed according to the finite element results to relieve the stress.

Socket Material Selection
The conventional molding of the socket is still using resin materials [11], and most of them are molded by plaster male molds. The advantages of resin are durability, light weight, high strength, and the characteristics of long-term use. The disadvantage is that the price is more expensive and the process is complicated. The advantage of thermoplastic sheet is that the material itself is flexible. During walking, the socket deforms according to the contraction of the soft tissue. The advantage is that it can reduce the force on the residual limbs, but the disadvantage is that it is not durable and needs to be replaced frequently, and there is a certain risk of allergies in direct contact. Therefore, this study still chooses resin materials for printing.

Socket cavity 3D printing process molding
Before 3D printing, it is necessary to set the parameters of the printed socket model. Including slice layer thickness, wall thickness, printing speed, printing height, retraction speed, moving speed, filling speed and other parameters. The setting of parameters will affect the overall printing effect and printing completion time. Due to the different conditions of the residual limbs, the specifications of the corresponding sockets are also different. Therefore, in terms of parameters such as printing speed and layer height, the current socket model 3D printing settings can be set for different sizes of sockets, and within the specification range, you can choose to print independently. The interface is simple, easy to learn, and both medical professionals and non-professionals can master it. Based on the surface of the prosthetic socket formed by Xingkang SLA8003D printing, the printed surface of the socket is relatively smooth, the molding accuracy is sufficient for use, and the overall strength of the socket is high. After the 3D printed socket is post-processed according to the caliber curve, the curved surface is relaxed and there is no warping phenomenon, as shown in Figure 16. On the basis of personalized design, the socket will be individually manufactured, and the use and wearing experience of the socket will be collected and verified in the follow-up study. The shape of the prosthesis in the section of the patient and the patient's walking experience will be observed on the basis of machine vision through a 3D scanner. Force posture, combined with the patient's previous gait to detect the best motion state, give the patient the best rehabilitation 3D prosthetic socket to restore most of the normal work and rest. Through the collection of a large number of patient data and corresponding personalized matching, systematic model construction is carried out, so that later patients can directly choose in the system.

Conclusion
This work proposes a 3D model construction system for personalized rehabilitation prostheses based on machine vision under the current situation of single medical rehabilitation tradition, which mainly conducts reasonable force modeling and analysis on the patient's section. Through the reverse design of the XTOM three-dimensional optical scanning measurement system under machine vision, the comparison between the three-dimensional digital model and the original design CAD data can ensure that the design result is consistent with the production accuracy, and the contour points on the surface of the human lower limb stump can be obtained through the three-dimensional scanner Cloud information in order to provide a basis for the design of the socket and the 3D printing model. Afterwards, research and analysis are carried out on the individual needs of patients, paving the way for the subsequent 3D modeling design. Before XKSLA800 industrial-grade 3D printing, the convolutional neural network and long-short-term memory network are used to extract the motion features of the patient's behavior, and then the prosthetic socket is modeled and analyzed for force, which conforms to the biomechanical distribution of the thigh stump. Properly modify the shape based on the finite element results above to prevent cracking and damage of the socket and relieve the stress. Finally, 3D printing is carried out on the selected materials and the corresponding prosthetic socket, and the printed object basically meets the design requirements of the scheme. There are still some small problems in our team's works, and there is little information on the number of patients' residual limbs. The following work will collect various patient residual faces in the hospital, and construct 3D prosthetics for rehabilitation to improve the system of the work.