Minimally invasive catheter-based approaches are gaining in importance in the treatment of cardiovascular diseases due its advantage in the clinical protocol. ARTERY is a European project that has the aim of providing the surgeon with a fully immersive augmented reality interface to monitor the intravascular and intracardiac scenarios of the intervention.
The direct and most important implication will be the removal of fluoroscopy that is a source of radiation field, harmful for both the patient and the surgeon. The need to have a real-time and extremely reliable computational simulation makes essential the role of the experimental validation.
The purpose of the current thesis is to build the proper experimental setup. The most relevant element is the phantom that must be suitable to replicate the behavior of the deformable vessel and the interaction with the catheter.

ARTERY European project focuses on the treatment of cardiovascular diseases, in particular the minimally invasive surgery perfomed with the Abbott catheter.
One of the possible applications is the repairment of the damaged mitral valve to prevent regurgitation. In this procedure, the catheter is used to place a clip (MitraClip) on the valve. The surgeon in this phase is guided by transesophageal echocardiographic videos, that he/she uses to localize the mitral valve and the clip.
This thesis aims to create a deep learning model for object detection, to automatize the localization of the valve and of the clip in the echocardiographic images. This tool will assist real time the operator to monitor the results of the procedure.

The use of needles in the field of minimally invasive procedures is nowadays a key tool that of offers a variety of options to surgeons.
In particular, steerable needles can be used in neurosurgery to achieve important tasks such as, for instance: Deep Brain Stimulation (DBS), drug deliveries, tumor ablations or biopsies.
The brain, being a very sensitive area, benefits a lot from the use of needles, that can more easily avoid damage to delicate structures, compared to standard surgery.
The implementation of this kind of procedure is nevertheless all but an easy task: the surgeon is provided with a lot of degrees of freedom: which kind of needle is it better to choose, which is the best entry point on the skull for the needle, which kind of feedback is given during the operation to perform an accurate steering of the needle.
Furthermore, the surgeon can rely on the use of path planning algorithms that can suggest the best trajectory for the catheter, if they are provided with some information like the maximum curvature achievable by the needle or which structures of the brain it should avoid or aim for.
The goal of this project is to create a software application able to help the neurosurgeon in the planning of the procedure, offering a set of already implemented features like brain models and path planning algorithms. The aim is to create an intuitive platform that could be used in a variety of situations and it will be qualitatively validated by a group of neurosurgeons, who will also be involved in the development process through questionnaires.

Robotic surgery is a type of Minimally Invasive Surgery (MIS) in which the surgeon should have motor skills alongside general surgical knowledge. So, the surgical trainee should have adequate training prior to the robotic laparoscopy. Nowadays, simulation is one of the most common ways of learning in many fields as well as surgery. They can help the trainee doctor to become ready for real-life exposure, improve the performance (high precision in less time), and reduce the training phase cost for the hospitals since simulations are done in laboratories.
The project aims to develop a training framework to obtain psychomotor skills for the surgical task while machine learning will allow the design of real-time personalized training feedback for each user depending on their motion.

Minimally Invasive Surgery (MIS) has quickly diffused in the field of cardiac interventions since it allows faster procedures and a shorter recovery time for patients. ATLAS is a European program for developing autonomous intraluminal, a challenging branch that appears in many MIS interventions and relies on steering flexible instruments through fragile lumens or vessels. This thesis aims to develop an integrated path planning approach that merges a learning-based algorithm, that uses demonstrations collected on a phantom, and the previous work (breadth first search – genetic algorithm). The results will then be rendered in a virtual simulator to provide real-time guidance through a mixed reality framework.

The Twin-to-Twin Transfusion Syndrome (TTTS) is a rare pathology that affects twin pregnancies with shared placenta. Its aetiology is the unequal blood flow along placental blood vessels known as anastomoses causing an high risk of perinatal mortality of one or both foetuses without any treatment. The recommended technique is Fetoscopic Minimally Invasive Surgery (MIS), that consists of a direct interruption of pathological anastomoses, via laser photo-coagulation. However, the identification of the inter-twin anastomoses is a challenging task due to different factors as the limited Field of View (FoV), the presence of the instrument and the large variability in the illumination level. The goal of this thesis is to provide Computer Assisted Intervention (CAI) support for the treatment by the detection of the pathologic anastomoses in intra-operative fetoscopic videos through Deep Learning methods. Moreover, the lack of confidence in the black-box nature of these algorithms and the weak supervision, lead to investigate also Explainable Artificial Intelligence (XAI) techniques, which aim at finding the set of domain features such as pixels of an image that contribute to the output decision of the model.

The surgical procedure needed to treat the twin-to-twin transfusion syndrome (TTTS) is difficult and requires very good skills. Nowadays, due to the low incidence of the syndrome, it is very difficult for the surgeon to perform easily the operation. The only way to train surgeons is to use a silicon probe which is very expensive. The goal of this thesis is to combine the deep learning and the virtual reality in order to develop an affordable virtual simulator as an alternative to the probe. The cross-platform Unity is used to develop the simulator, including all the devices needed for the intervention like the endoscope, the camera and the laser fibre. In order to reproduce the variability and obtain a more realistic placenta-background for the simulator, an image-to-image translation domain is used. In particular, the translation domain will be performed by a cycleGAN network, which will generete images of in-vivo placenta starting from ex-vivo ones.

Minimally invasive catheter-based approaches are gaining in importance in the treatment of cardiovascular diseases due its advantage in the clinical protocol. ARTERY European project proposes, in this context, a radiation-free approach, where the operator, through a fully immersive augmented reality interface, will be able to monitor the intravascular and intracardiac positioning of the catheter.
This thesis deals with the automatization of the set of actions, nowadays executed by surgeon, that has to be performed by the actuators, positioned inside the stabilizer of the catheter, in order to move the catheter itself. After the CAD modeling of the stabilizer, the actuators of the given steerable catheter will be governed by the Arduino system. Moreover, it is needed to use ROS (i.e. Robot Operating System) so that it will possible to integrate all the signals coming from the sensors mounted on the catheter and the constraints of the catheter with the actuation command: in this way, it will be possible to control the catheter and perform the desired trajectory of the surgeon.

In the last years minimally invasive catheter-based approaches are gaining in importance for the treatment of patient who cannot be subject to conventionally therapy due to high risks. ARTERY European project offers a fully immersive augmented reality interface by which the operator will monitor the intravascular path of the catheter with no need for radiation-based imaging. The operator will define the catheter target pose by simple gesture into the holographic representation. The aim of this thesis is to develop a kinematic and dynamic model of the catheter, considering its constraints, in order to translate the surgeon input into a set of actions to be performed to reach the desired catheter pose.

Cardiovascular diseases are the single most common cause of death in Europe and worldwide. Minimally invasive catheter-based approaches are gaining in importance in the treatment of these diseases.
ARTERY European project proposes an improvement in the treatment of cadiovascular diseases thanks to its radiation free nature, its minimally invasive surgery and through the use of augmented reality.
New path planning methods have improved the accuracy and the precision of the catheter to reach a defined target given a starting point. This thesis is part of ARTERY European project and it aims to obtain both the preoperative and intraoperative path planning, both for intracardiac and intravascular, of the steerable needle. Path planning is obtained by developing a model through machine learning, which finds the optimal path taking into account constraints given by the anatomical and physical structure of the vessels and the heart, dictated by the flexible needle’s kinematics. In the intravascular phase the needle has to be as far as possible from the vessels’ walls, so it does not damage them, while in the intracardiac phase it has to find the best positioning for the tip.
In both cases the agent should be able to compute the path satisfying constraints and avoiding obstacles while going from a starting point to the target.

The introduction of Robotic-Assisted Minimally Invasive Surgery has been helping the surgeons performing different endoscopy and laparoscopy procedure while reducing the incidence of surgical complications. Furthermore, the assisted surgeon gains a greater range of motion and dexterity, a better access to the interested area and he is able to see a high-resolution image of the operating field. However, the lack of haptic feedback is a key limiting factor and it can cause an increase of intra-operative injuries. The goal of this thesis is to develop a robot-assisted palpation system by exploiting the haptic feedback to get information about the different stiffness of the tissues. Doing so, the haptic feedback can be used in important decision-making scenarios such as the discrimination of healthy versus tumoral tissues. This is achieved by the implementation and comparison of different control strategies in order to assure stability and transparency.

Augmented Reality offers an innovative approach for treatments, education, and surgeries.
AR in minimally invasive surgery aims to provide to the surgeon a more comfortable environment in which work by improving the quality of navigation, mainly in situation where a lack of visibility strongly limits the sight, moreover in current vision system a limiting factor is that visualization is performed in 2D.
The goal of this project is to connect an augmented reality device, composed of a Head-mounted display and controllers, to a camera of the endoscope in order to visualize in AR what is captured by the camera with, in addition, some virtual information in order to enrich the experience and so improve speed and accuracy.
Thanks to the AR technologies, it is possible provide 3D vision that brings information of depth, guidance through optimized trajectories, information about tissues, visual feedback due to interactions, and so refine the performances.

Reaching deep located targets of brain still has a major challenge for keyhole neurosurgery. The advanced 3D path planning methods have greatly improved the possibilities for complex robotic assisted neurosurgery, where the complexity and high sensitivity of the anatomical regions requires fine precision and dexterity. EDEN2020 european project designed steerable catheter lead to developing new path planning approaches thanks to its segmented structure. The thesis is aiming pre-operative planning of the needle path by developing a model free reinforcement learning and finding the optimal path which takes into account constraints coming from anatomical obstacles and physical constraints dictated by flexible needle kinematics. Thus in the context of RL method, the agent does optimal action which means getting maximum feedback from each state to minimize cost of action. In Asynchronous Advantage Actor-Critic (A3C), multiple agents play concurrently and optimize a deep neural network controller using asynchronous gradient descent and it feeds agent experiences directly into training. The proposed method aims to obtain an higher accuracy with a reasonably lower computational time.

Minimally invasive neurosurgery presents many advantages over conventional neurosurgical approaches as a reduced patient trauma and a shorter recovery time. The spread of this type of procedures, in particular of the so-called keyhole neurosurgery, has spurred the development of steerable needles as they can reach targets behind sensitive or impenetrable areas. EDEN2020 is a European project which has the ambition to set the standard for one step neurosurgical diagnostics and therapy through a minimally invasive approach. The final aim of this thesis is the control of the EDEN2020 steerable needle through the integration of the Unity simulator and the framework which can directly steer the catheter (Robot Operating System/ROS). The simulator provides the trajectory to follow in order to reach the target avoiding obstacles while ROS enables the execution of the real catheter movement and returns the actual tip position. The control model estimates the error between the aforesaid and the real tip positions and allows the formulation of a new trajectory taking into consideration the catheter deformation, the brain deformation and the catheter delay.