Minimally invasive surgery (MIS) involves a multidimensional series of tasks requiring a synthesis between visual information and the kinematics and dynamics of the surgical tools. Analysis of these sources of information is a key step in mastering MIS but may also be used to define objective criteria for characterizing surgical performance. The BlueDRAGON is a new system for acquiring the kinematics and the dynamics of two endoscopic tools along synchronized with the visual view of the surgical scene. It includes two four-bar passive mechanisms equipped with position and force torque sensors for measuring the positions and the orientations (P/O) of two endoscopic tools along with the forces and torques (F/T) applied by the surgeon's hands. The methodology of decomposing the surgical task is based on a fully connected, 28 finite-states Markov model where each states corresponded to a fundamental tool/tissue interaction based on the tool kinematics and associated with unique F/T signatures. The experimental protocol included seven MIS tasks performed on an animal model (pig) by 30 surgeons at different levels of their residency training including expert surgeons. Preliminary analysis of these data showed that major differences between residents at different skill levels were: (/) the types of tool/tissue interactions being used, (i/) the transitions between tool/tissue interactions being applied by each hand, (iii) time spent while performing each tool/tissue interaction, (iv) the overall completion time, and (v) the variable F/T magnitudes being applied by the subjects through the endoscopic tools. Systems like surgical robots or virtual reality simulators that inherently measure the kinematics and the dynamics of the surgical tool may benefit from inclusion of the prop...

Abstract—Minimally invasive surgery (MIS) involves a multidimensional series of tasks requiring a synthesis between visual information and the kinematics and dynamics of the surgical tools. Analysis of these sources of information is a key step in defining objective criteria for characterizing surgical performance. The Blue DRAGON is a new system for acquiring the kinematics and the dynamics of two endoscopic tools synchronized with the endoscopic view of the surgical scene. Modeling the process of MIS using a finite state model [Markov model (MM)] reveals the internal structure of the surgical task and is utilized as one of the key steps in objectively assessing surgical performance. The experimental protocol includes tying an intracorporeal knot in a MIS setup performed on an animal model (pig) by 30 surgeons at different levels of training including expert surgeons. An objective learning curve was defined based on measuring quantitative statistical distance (similarity) between MM of experts and MM of residents at different levels of training. The objective learning curve was similar to that of the subjective performance analysis. The MM proved to be a powerful and compact mathematical model for decomposing a complex task such as laparoscopic suturing. Systems like surgical robots or virtual reality simulators in which the kinematics and the dynamics of the surgical tool are inherently measured may benefit from incorporation of the proposed methodology. Index Terms—Dynamics, haptics, human machine interface, kinematics, manipulation, Markov model, minimally invasive, robotics, simulation, soft tissue, surgery, surgical skill assessment, surgical tool, vector quantization. I.

Minimally invasive surgery (MIS) involves a multi-dimensional series of tasks requiring a synthesis between visual information and the kinematics and dynamics of the surgical tools. Analysis of these sources of information is a key step in mastering MIS surgery but may also be used to define objective criteria for characterizing surgical performance. The BlueDRAGON is a new system for acquiring the kinematics and the dynamics of two endoscopic tools along with the visual view of the surgical scene. It includes two four-bar mechanisms equipped with position and force torque sensors for measuring the positions and the orientations (P/O) of two endoscopic tools along with the forces and torques applied by the surgeon's hands. The methodology of decomposing the surgical task is based on a fully connected, finite-states (28 states) Markov model where each states corresponded to a fundamental tool/tissue interaction based on the tool kinematics and associated with unique F/T signatures. The experimental protocol included seven MIS tasks performed on an animal model (pig) by 30 surgeons at different levels of their residency training. Preliminary analysis of these data showed that major differences between residents at different skill levels were: (i) the types of tool/tissue interactions being used, (ii) the transitions between tool/tissue interactions being applied by each hand, (iii) time spent while performing each tool/tissue interaction, (iv) the overall completion time, and (v) the variable F/T magnitudes being applied by the subjects through the endoscopic tools. Systems like surgical robots or virtual reality simulators that inherently measure the kinematics and the dynamics of the surgical tool may benefit from inclusion of the proposed methodolo...

Objective: Evaluation of the laparoscopic surgical skills of surgical residents is usually a subjective process carried out in the operating room by senior surgeons. The two hypotheses of the current study were: (1) haptic information and tool/tissue interactions (types and transitions) performed in laparoscopic surgery are skill-dependent, and (2) statistical models (Hidden Markov Models---HMMs) incorporating these data are capable of objectively evaluating laparoscopic surgical skills.

Surgical robotic systems and virtual reality simulators have introduced an unprecedented precision of measurement for both tool-tissue and toolsurgeon interaction; thus holding promise for more objective analyses of surgical skill. Integrative or averaged metrics such as path length, timeto -task, success/failure percentages, etc., have often been employed towards this end but these fail to address the processes associated with a surgical task as a dynamic phenomena. Stochastic tools such as Markov modeling using a `white-box' approach have proven amenable to this type of analysis. While such an approach reveals the internal structure of the of the surgical task as a process, it requires a task decomposition based on expert knowledge, which may result in a relatively large/complex model. In this work, a `black box' approach is developed with generalized cross-procedural applications., the model is characterized by a compact topology, abstract state definitions, and optimized codebook size. Data sets of isolated tasks were extracted from the Blue DRAGON database consisting of 30 surgical subjects stratified into six training levels. Vector quantization (VQ) was employed on the entire database, thus synthesizing a lexicon of discrete, task-independent surgical tool/tissue interactions. VQ has successfully established a dictionary of 63 surgical code words and displayed non-temporal skill discrimination. VQ allows for a more cross-procedural analysis without relying on a thorough study of the procedure, links the results of the black-box approach to observable phenomena, and reduces the computational cost of the analysis by discretizing a complex, continuous data space.

Today, robotic devices are used to replace missing limbs, perform delicate surgical procedures, deliver neurorehabilitation therapy to stroke patients, teach children with learning disabilities, and perform a growing number

Abstract — In the future, robotic surgical assistants may assist surgeons by performing specific subtasks such as retraction and suturing to reduce surgeon tedium and reduce the duration of some operations. We propose an apprenticeship learning approach that has potential to allow robotic surgical assistants to autonomously execute specific trajectories with superhuman performance in terms of speed and smoothness. In the first step, we record a set of trajectories using human-guided backdriven motions of the robot. These are then analyzed to extract a smooth reference trajectory, which we execute at gradually increasing speeds using a variant of iterative learning control. We evaluate this approach on two representative tasks using the Berkeley Surgical Robots: a figure eight trajectory and a two handed knot-tie, a tedious suturing sub-task required in many surgical procedures. Results suggest that the approach enables (i) rapid learning of trajectories, (ii) smoother trajectories than the human-guided trajectories, and (iii) trajectories that are 7 to 10 times faster than the best human-guided trajectories. I.

Abstract — Conservation of historic documents is often necessary to preserve their cultural value for future generations. An important component of the skill of document conservation is delicate hands-on manipulations. As with medical procedural training, there is a great need for better ways to train document conservators in these skills. This paper reports initial measurements of forces and torques at the interaction point between tools and mocked up documents. Five conservators used their preferred tools (such as scalpel, needle, brush, and microspatula) to remove material adhered to the samples. We analyzed video and 100 Hz force and torque recordings in the time and frequency domain to gain understanding of the nature of these tasks. The results can inform design of training simulators for document conservation skills. I.

Laparoscopic surgical training is a challenging task due to the complexity of instrument control and demand on manual dexterity and hand-eye coordination. Currently, training and assessing surgeons for their laparoscopic skills rely mainly on subjective assessment. This paper presents a Body Sensor Network (BSN) sensor glove for laparoscopic gesture recognition and objective assessment of surgical skills. An HMM framework is proposed for the selection of sensors to maximize the sensitivity and specificity of gesture recognition for a given set of laparoscopic tasks. With the proposed framework, the optimal location as well as the number of the sensors can be determined. The sensors used in this study include accelerometers and fiber optic bend sensors. Experimental data is collected by participants wearing the glove while performing simple laparoscopic tasks. By using the proposed HMM framework, sensor correlation and relevance to task recognition can be determined, thus allowing a reduction in the number of sensors used. Results have shown that it is possible to establish the intrinsic correlation of the sensors and determine which sensors are most relevant to specific gestures based on the proposed method.

Abstract — Recent advances in the modeling of deformable one-dimensional objects (DOOs) such as surgical suture, rope, and hair show significant promise for improving the simulation, perception, and manipulation of such objects. An important application of these tasks lies in the area of medical robotics, where robotic surgical assistants have the potential to greatly reduce surgeon fatigue and human error by improving the accuracy, speed, and robustness of surgical tasks such as suturing. However, different types of DOOs exhibit a variety of bending and twisting behaviors that are highly dependent on material properties. This paper proposes an approach for fitting simulation models of DOOs to observed data. Our approach learns an energy function such that observed DOO configurations lie in local energy minima. Our experiments on a variety of DOOs show that models fitted to different types of DOOs using our approach enable accurate prediction of future configurations. Additionally, we explore the application of our learned model to the perception of DOOs. I.