Header logo is


2017


Thumb xl pyramid
Optical Flow Estimation using a Spatial Pyramid Network

Ranjan, A., Black, M.

In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (inproceedings)

Abstract
We learn to compute optical flow by combining a classical spatial-pyramid formulation with deep learning. This estimates large motions in a coarse-to-fine approach by warping one image of a pair at each pyramid level by the current flow estimate and computing an update to the flow. Instead of the standard minimization of an objective function at each pyramid level, we train one deep network per level to compute the flow update. Unlike the recent FlowNet approach, the networks do not need to deal with large motions; these are dealt with by the pyramid. This has several advantages. First, our Spatial Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms of model parameters. This makes it more efficient and appropriate for embedded applications. Second, since the flow at each pyramid level is small (< 1 pixel), a convolutional approach applied to pairs of warped images is appropriate. Third, unlike FlowNet, the learned convolution filters appear similar to classical spatio-temporal filters, giving insight into the method and how to improve it. Our results are more accurate than FlowNet on most standard benchmarks, suggesting a new direction of combining classical flow methods with deep learning.

ps

pdf SupMat project/code [BibTex]

2017


pdf SupMat project/code [BibTex]


Thumb xl web teaser
Detailed, accurate, human shape estimation from clothed 3D scan sequences

Zhang, C., Pujades, S., Black, M., Pons-Moll, G.

In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, Washington, DC, USA, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), July 2017, Spotlight (inproceedings)

Abstract
We address the problem of estimating human body shape from 3D scans over time. Reliable estimation of 3D body shape is necessary for many applications including virtual try-on, health monitoring, and avatar creation for virtual reality. Scanning bodies in minimal clothing, however, presents a practical barrier to these applications. We address this problem by estimating body shape under clothing from a sequence of 3D scans. Previous methods that have exploited statistical models of body shape produce overly smooth shapes lacking personalized details. In this paper we contribute a new approach to recover not only an approximate shape of the person, but also their detailed shape. Our approach allows the estimated shape to deviate from a parametric model to fit the 3D scans. We demonstrate the method using high quality 4D data as well as sequences of visual hulls extracted from multi-view images. We also make available a new high quality 4D dataset that enables quantitative evaluation. Our method outperforms the previous state of the art, both qualitatively and quantitatively.

ps

arxiv_preprint video dataset pdf supplemental [BibTex]

arxiv_preprint video dataset pdf supplemental [BibTex]


Thumb xl imgidx 00197
Multiple People Tracking by Lifted Multicut and Person Re-identification

Tang, S., Andriluka, M., Andres, B., Schiele, B.

In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages: 3701-3710, IEEE Computer Society, Washington, DC, USA, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (inproceedings)

ps

DOI [BibTex]

DOI [BibTex]


Thumb xl vpn teaser
Video Propagation Networks

Jampani, V., Gadde, R., Gehler, P. V.

In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (inproceedings)

ps

pdf supplementary arXiv project page code [BibTex]

pdf supplementary arXiv project page code [BibTex]


no image
Dynamic Time-of-Flight

Schober, M., Adam, A., Yair, O., Mazor, S., Nowozin, S.

Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, pages: 170-179, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (conference)

ei pn

DOI [BibTex]

DOI [BibTex]


Thumb xl cvpr2017 landpsace
Semantic Multi-view Stereo: Jointly Estimating Objects and Voxels

Ulusoy, A. O., Black, M. J., Geiger, A.

In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (inproceedings)

Abstract
Dense 3D reconstruction from RGB images is a highly ill-posed problem due to occlusions, textureless or reflective surfaces, as well as other challenges. We propose object-level shape priors to address these ambiguities. Towards this goal, we formulate a probabilistic model that integrates multi-view image evidence with 3D shape information from multiple objects. Inference in this model yields a dense 3D reconstruction of the scene as well as the existence and precise 3D pose of the objects in it. Our approach is able to recover fine details not captured in the input shapes while defaulting to the input models in occluded regions where image evidence is weak. Due to its probabilistic nature, the approach is able to cope with the approximate geometry of the 3D models as well as input shapes that are not present in the scene. We evaluate the approach quantitatively on several challenging indoor and outdoor datasets.

avg ps

YouTube pdf suppmat [BibTex]

YouTube pdf suppmat [BibTex]


Thumb xl judith
Deep representation learning for human motion prediction and classification

Bütepage, J., Black, M., Kragic, D., Kjellström, H.

In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (inproceedings)

Abstract
Generative models of 3D human motion are often restricted to a small number of activities and can therefore not generalize well to novel movements or applications. In this work we propose a deep learning framework for human motion capture data that learns a generic representation from a large corpus of motion capture data and generalizes well to new, unseen, motions. Using an encoding-decoding network that learns to predict future 3D poses from the most recent past, we extract a feature representation of human motion. Most work on deep learning for sequence prediction focuses on video and speech. Since skeletal data has a different structure, we present and evaluate different network architectures that make different assumptions about time dependencies and limb correlations. To quantify the learned features, we use the output of different layers for action classification and visualize the receptive fields of the network units. Our method outperforms the recent state of the art in skeletal motion prediction even though these use action specific training data. Our results show that deep feedforward networks, trained from a generic mocap database, can successfully be used for feature extraction from human motion data and that this representation can be used as a foundation for classification and prediction.

ps

arXiv [BibTex]

arXiv [BibTex]


Thumb xl teasercrop
Unite the People: Closing the Loop Between 3D and 2D Human Representations

Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., Gehler, P. V.

In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (inproceedings)

Abstract
3D models provide a common ground for different representations of human bodies. In turn, robust 2D estimation has proven to be a powerful tool to obtain 3D fits “in-the-wild”. However, depending on the level of detail, it can be hard to impossible to acquire labeled data for training 2D estimators on large scale. We propose a hybrid approach to this problem: with an extended version of the recently introduced SMPLify method, we obtain high quality 3D body model fits for multiple human pose datasets. Human annotators solely sort good and bad fits. This procedure leads to an initial dataset, UP-3D, with rich annotations. With a comprehensive set of experiments, we show how this data can be used to train discriminative models that produce results with an unprecedented level of detail: our models predict 31 segments and 91 landmark locations on the body. Using the 91 landmark pose estimator, we present state-of-the art results for 3D human pose and shape estimation using an order of magnitude less training data and without assumptions about gender or pose in the fitting procedure. We show that UP-3D can be enhanced with these improved fits to grow in quantity and quality, which makes the system deployable on large scale. The data, code and models are available for research purposes.

ps

arXiv project/code/data [BibTex]

arXiv project/code/data [BibTex]


Thumb xl anja
Generating Descriptions with Grounded and Co-Referenced People

Rohrbach, A., Rohrbach, M., Tang, S., Oh, S. J., Schiele, B.

In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages: 4196-4206, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (inproceedings)

ps

PDF DOI [BibTex]

PDF DOI [BibTex]


Thumb xl fig1
Locomotion of light-driven soft microrobots through a hydrogel via local melting

Palagi, S., Mark, A. G., Melde, K., Qiu, T., Zeng, H., Parmeggiani, C., Martella, D., Wiersma, D. S., Fischer, P.

In 2017 International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS), pages: 1-5, July 2017 (inproceedings)

Abstract
Soft mobile microrobots whose deformation can be directly controlled by an external field can adapt to move in different environments. This is the case for the light-driven microrobots based on liquid-crystal elastomers (LCEs). Here we show that the soft microrobots can move through an agarose hydrogel by means of light-controlled travelling-wave motions. This is achieved by exploiting the inherent rise of the LCE temperature above the melting temperature of the agarose gel, which facilitates penetration of the microrobot through the hydrogel. The locomotion performance is investigated as a function of the travelling-wave parameters, showing that effective propulsion can be obtained by adapting the generated motion to the specific environmental conditions.

pf

DOI [BibTex]

DOI [BibTex]


Thumb xl mosh heroes icon
Method for providing a three dimensional body model

Loper, M., Mahmood, N., Black, M.

U.S. Patent 9,710,964 B2., July 2017 (patent)

Abstract
A method for providing a three-dimensional body model which may be applied for an animation, based on a moving body, wherein the method comprises providing a parametric three-dimensional body model, which allows shape and pose variations; applying a standard set of body markers; optimizing the set of body markers by generating an additional set of body markers and applying the same for providing 3D coordinate marker signals for capturing shape and pose of the body and dynamics of soft tissue; and automatically providing an animation by processing the 3D coordinate marker signals in order to provide a personalized three-dimensional body model, based on estimated shape and an estimated pose of the body by means of predicted marker locations.

ps

Google Patents MoSh Project [BibTex]


no image
Strategic exploration in human adaptive control

Schulz, E., Klenske, E., Bramley, N., Speekenbrink, M.

Proceedings of the 39th Annual Conference of the Cognitive Science Society (CogSci), (Editors: Glenn Gunzelmann, Andrew Howes, Thora Tenbrink and Eddy J. Davelaar), cognitivesciencesociety.org, July 2017 (conference)

ei

link (url) [BibTex]

link (url) [BibTex]


no image
Event-based State Estimation: An Emulation-based Approach

Trimpe, S.

IET Control Theory & Applications, 11(11):1684-1693, July 2017 (article)

Abstract
An event-based state estimation approach for reducing communication in a networked control system is proposed. Multiple distributed sensor agents observe a dynamic process and sporadically transmit their measurements to estimator agents over a shared bus network. Local event-triggering protocols ensure that data is transmitted only when necessary to meet a desired estimation accuracy. The event-based design is shown to emulate the performance of a centralised state observer design up to guaranteed bounds, but with reduced communication. The stability results for state estimation are extended to the distributed control system that results when the local estimates are used for feedback control. Results from numerical simulations and hardware experiments illustrate the effectiveness of the proposed approach in reducing network communication.

am ics

arXiv Supplementary material PDF DOI Project Page [BibTex]

arXiv Supplementary material PDF DOI Project Page [BibTex]


no image
Learning Movement Primitive Libraries through Probabilistic Segmentation

Lioutikov, R., Neumann, G., Maeda, G., Peters, J.

International Journal of Robotics Research, 36(8):879-894, July 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Asymptotic Normality of the Median Heuristic

Garreau, Damien

July 2017, preprint (unpublished)

link (url) [BibTex]

link (url) [BibTex]


no image
Design of a Parallel Continuum Manipulator for 6-DOF Fingertip Haptic Display

Young, E. M., Kuchenbecker, K. J.

In Proceedings of the IEEE World Haptics Conference (WHC), pages: 599-604, June 2017, Finalist for best poster paper (inproceedings)

Abstract
Despite rapid advancements in the field of fingertip haptics, rendering tactile cues with six degrees of freedom (6 DOF) remains an elusive challenge. In this paper, we investigate the potential of displaying fingertip haptic sensations with a 6-DOF parallel continuum manipulator (PCM) that mounts to the user's index finger and moves a contact platform around the fingertip. Compared to traditional mechanisms composed of rigid links and discrete joints, PCMs have the potential to be strong, dexterous, and compact, but they are also more complicated to design. We define the design space of 6-DOF parallel continuum manipulators and outline a process for refining such a device for fingertip haptic applications. Following extensive simulation, we obtain 12 designs that meet our specifications, construct a manually actuated prototype of one such design, and evaluate the simulation's ability to accurately predict the prototype's motion. Finally, we demonstrate the range of deliverable fingertip tactile cues, including a normal force into the finger and shear forces tangent to the finger at three extreme points on the boundary of the fingertip.

hi

DOI [BibTex]

DOI [BibTex]


no image
Handling Scan-time Parameters in Haptic Surface Classification

Burka, A., Kuchenbecker, K. J.

In Proceedings of the IEEE World Haptics Conference (WHC), pages: 424-429, June 2017 (inproceedings)

hi

DOI [BibTex]

DOI [BibTex]


no image
High Magnitude Unidirectional Haptic Force Display Using a Motor/Brake Pair and a Cable

Hu, S., Kuchenbecker, K. J.

In Proceedings of the IEEE World Haptics Conference (WHC), pages: 394-399, June 2017 (inproceedings)

Abstract
Clever electromechanical design is required to make the force feedback delivered by a kinesthetic haptic interface both strong and safe. This paper explores a onedimensional haptic force display that combines a DC motor and a magnetic particle brake on the same shaft. Rather than a rigid linkage, a spooled cable connects the user to the actuators to enable a large workspace, reduce the moving mass, and eliminate the sticky residual force from the brake. This design combines the high torque/power ratio of the brake and the active output capabilities of the motor to provide a wider range of forces than can be achieved with either actuator alone. A prototype of this device was built, its performance was characterized, and it was used to simulate constant force sources and virtual springs and dampers. Compared to the conventional design of using only a motor, the hybrid device can output higher unidirectional forces at the expense of free space feeling less free.

hi

DOI [BibTex]

DOI [BibTex]


Thumb xl publications toc
Dynamic analysis on hexapedal water-running robot with compliant joints

Kim, H., Liu, Y., Jeong, K., Sitti, M., Seo, T.

In 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), pages: 250-251, June 2017 (inproceedings)

Abstract
The dynamic analysis has been considered as one of the important design methods to design robots. In this research, we derive dynamic equation of hexapedal water-running robot to design compliant joints. The compliant joints that connect three bodies will be used to improve mobility and stability of water-running motion's pitch behavior. We considered all of parts as rigid body including links of six Klann mechanisms and three main frames. And then, we derived dynamic equation by using the Lagrangian method with external force of the water. We are expecting that the dynamic analysis is going to be used to design parts of the water running robot.

pi

DOI [BibTex]

DOI [BibTex]


Thumb xl imahe toc
Soiled adhesive pads shear clean by slipping: a robust self-cleaning mechanism in climbing beetles

Amador, G., Endlein, T., Sitti, M.

Journal of The Royal Society Interface, 14(131):20170134, The Royal Society, June 2017 (article)

Abstract
Animals using adhesive pads to climb smooth surfaces face the problem of keeping their pads clean and functional. Here, a self-cleaning mechanism is proposed whereby soiled feet would slip on the surface due to a lack of adhesion but shed particles in return. Our study offers an in situ quantification of self-cleaning performance in fibrillar adhesives, using the dock beetle as a model organism. After beetles soiled their pads by stepping into patches of spherical beads, we found that their gait was significantly affected. Specifically, soiled pads slipped 10 times further than clean pads, with more particles deposited for longer slips. Like previous studies, we found that particle size affected cleaning performance. Large (45 μm) beads were removed most effectively, followed by medium (10 μm) and small (1 μm). Consistent with our results from climbing beetles, force measurements on freshly severed legs revealed larger detachment forces of medium particles from adhesive pads compared to a flat surface, possibly due to interlocking between fibres. By contrast, dock leaves showed an overall larger affinity to the beads and thus reduced the need for cleaning. Self-cleaning through slippage provides a mechanism robust to particle size and may inspire solutions for artificial adhesives.

pi

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Thumb xl comp 3e 00000 copy
Non-Equilibrium Assembly of Light-Activated Colloidal Mixtures

Singh, D. P., Choudhury, U., Fischer, P., Mark, A. G.

Advanced Materials, 29, pages: 1701328, June 2017, 32 (article)

Abstract
The collective phenomena exhibited by artificial active matter systems present novel routes to fabricating out-of-equilibrium microscale assemblies. Here, the crystallization of passive silica colloids into well-controlled 2D assemblies is shown, which is directed by a small number of self-propelled active colloids. The active colloids are titania–silica Janus particles that are propelled when illuminated by UV light. The strength of the attractive interaction and thus the extent of the assembled clusters can be regulated by the light intensity. A remarkably small number of the active colloids is sufficient to induce the assembly of the dynamic crystals. The approach produces rationally designed colloidal clusters and crystals with controllable sizes, shapes, and symmetries. This multicomponent active matter system offers the possibility of obtaining structures and assemblies that cannot be found in equilibrium systems.

pf

link (url) DOI [BibTex]


no image
Yield prediction in parallel homogeneous assembly

Ipparthi, D., Winslow, A., Sitti, M., Dorigo, M., Mastrangeli, M.

Soft Matter, 13, pages: 7595-7608, The Royal Society of Chemistry, June 2017 (article)

Abstract
We investigate the parallel assembly of two-dimensional{,} geometrically-closed modular target structures out of homogeneous sets of macroscopic components of varying anisotropy. The yield predicted by a chemical reaction network (CRN)-based model is quantitatively shown to reproduce experimental results over a large set of conditions. Scaling laws for parallel assembling systems are then derived from the model. By extending the validity of the CRN-based modelling{,} this work prompts analysis and solutions to the incompatible substructure problem.

pi

link (url) DOI [BibTex]

link (url) DOI [BibTex]


Thumb xl kim et al 2017 advanced materials
Nanodiamonds That Swim

Kim, J. T., Choudhury, U., Hyeon-Ho, J., Fischer, P.

Advanced Materials, 29(30):1701024, June 2017, Back Cover (article)

Abstract
Nanodiamonds are emerging as nanoscale quantum probes for bio-sensing and imaging. This necessitates the development of new methods to accurately manipulate their position and orientation in aqueous solutions. The realization of an “active” nanodiamond (ND) swimmer in fluids, composed of a ND crystal containing nitrogen vacancy centers and a light-driven self-thermophoretic micromotor, is reported. The swimmer is propelled by a local temperature gradient created by laser illumination on its metal-coated side. Its locomotion—from translational to rotational motion—is successfully controlled by shape-dependent hydrodynamic interactions. The precise engineering of the swimmer's geometry is achieved by self-assembly combined with physical vapor shadow growth. The optical addressability of the suspended ND swimmers is demonstrated by observing the electron spin resonance in the presence of magnetic fields. Active motion at the nanoscale enables new sensing capabilities combined with active transport including, potentially, in living organisms.

pf

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Three‐dimensional patterning in biomedicine: Importance and applications in neuropharmacology

Ajay, V. S., Tanmay, G., Madu, B., Byung‐Wook, P., Thomas, E., Metin, S.

Journal of Biomedical Materials Research Part B: Applied Biomaterials, 106(3):1369-1382, June 2017 (article)

Abstract
Abstract Nature manufactures biological systems in three dimensions with precisely controlled spatiotemporal profiles on hierarchical length and time scales. In this article, we review 3D patterning of biological systems on synthetic platforms for neuropharmacological applications. We briefly describe 3D versus 2D chemical and topographical patterning methods and their limitations. Subsequently, an overview of introducing a third dimension in neuropharmacological research with delineation of chemical and topographical roles is presented. Finally, toward the end of this article, an explanation of how 3D patterning has played a pivotal role in relevant fields of neuropharmacology to understand neurophysiology during development, normal health, and disease conditions is described. The future prospects of organs‐on‐a‐‐like devices to mimic patterned blood–brain barrier in the context of neurotherapeutic discovery and development for the prioritization of lead candidates, membrane potential, and toxicity testing are also described. © 2017 Wiley Periodicals, Inc. J Biomed Mater Res Part B: Appl Biomater, 106B: 1369–1382, 2018.

pi

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Physically Interactive Exercise Games with a Baxter Robot

Fitter, N. T., Kuchenbecker, K. J.

Hands-on demonstration presented at the 2017 IEEE World Haptics Conference (WHC), June 2017 (misc)

hi

[BibTex]

[BibTex]


no image
Perception of force and stiffness in the presence of low-frequency haptic noise

Gurari, N., Okamura, A. M., Kuchenbecker, K. J.

PLoS ONE, 12(6):e0178605, June 2017 (article)

hi

[BibTex]

[BibTex]


no image
A Wrist-Squeezing Force-Feedback System for Robotic Surgery Training

Brown, J. D., Fernandez, J. N., Cohen, S. P., Kuchenbecker, K. J.

In Proceedings of the IEEE World Haptics Conference (WHC), pages: 107-112, June 2017 (inproceedings)

Abstract
Over time, surgical trainees learn to compensate for the lack of haptic feedback in commercial robotic minimally invasive surgical systems. Incorporating touch cues into robotic surgery training could potentially shorten this learning process if the benefits of haptic feedback were sustained after it is removed. In this paper, we develop a wrist-squeezing haptic feedback system and evaluate whether it holds the potential to train novice da Vinci users to reduce the force they exert on a bimanual inanimate training task. Subjects were randomly divided into two groups according to a multiple baseline experimental design. Each of the ten participants moved a ring along a curved wire nine times while the haptic feedback was conditionally withheld, provided, and withheld again. The realtime tactile feedback of applied force magnitude significantly reduced the integral of the force produced by the da Vinci tools on the task materials, and this result remained even when the haptic feedback was removed. Overall, our findings suggest that wrist-squeezing force feedback can play an essential role in helping novice trainees learn to minimize the force they exert with a surgical robot.

hi

DOI [BibTex]

DOI [BibTex]


no image
Proton Pack: Visuo-Haptic Surface Data Recording

Burka, A., Kuchenbecker, K. J.

Hands-on demonstration presented at the 2017 IEEE World Haptics Conference (WHC), June 2017 (misc)

hi

[BibTex]

[BibTex]


Thumb xl 07873311 toc
Tail-Assisted Mobility and Stability Enhancement in Yaw/Pitch Motions of a Water-Running Robot

Kim, H., Sitti, M., Seo, T.

IEEE/ASME Transactions on Mechatronics, 22, IEEE, June 2017 (article)

Abstract
Water-running robots have been developed inspired by a basilisk lizard, which demonstrates highly agile, stable, and energy-efficient locomotion on water surfaces. Current water-running robots are not as stable and agile as their biological counterparts. This study shows how the stability of a water-running robot in the pitch direction can be improved by using an active tail to enable increased propulsion. The mobility of the robot is also increased. To generate force in the pitch and yaw directions, a two-degrees-of-freedom tail is implemented with two circular plates to provide drag. We developed two types of dynamic models for pitch and yaw behavior, and the results are recursively calculated by considering the correlation between the models. The relationship between pitch motion and propulsion was analyzed by simulations. The steering behavior of the robot is also validated while considering the pitch behavior. Experiments were conducted to verify the simulation results.

pi

DOI [BibTex]

DOI [BibTex]


no image
State-Regularized Policy Search for Linearized Dynamical Systems

Abdulsamad, H., Arenz, O., Peters, J., Neumann, G.

Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling, (ICAPS), pages: 419-424, (Editors: Laura Barbulescu, Jeremy Frank, Mausam and Stephen F. Smith), AAAI Press, June 2017 (conference)

ei

link (url) [BibTex]

link (url) [BibTex]


no image
Teaching a Robot to Collaborate with a Human Via Haptic Teleoperation

Hu, S., Kuchenbecker, K. J.

Work-in-progress paper (2 pages) presented at the 2017 IEEE World Haptics Conference (WHC), June 2017 (misc)

hi

Project Page [BibTex]

Project Page [BibTex]


Thumb xl full outfit
How Should Robots Hug?

Block, A. E., Kuchenbecker, K. J.

Work-in-progress paper (2 pages) presented at the 2017 IEEE World Haptics Conference (WHC), June 2017 (misc)

hi

[BibTex]

[BibTex]


no image
Evaluation of a vibrotactile simulator for dental caries detection

Kuchenbecker, K. J., Parajon, R., Maggio, M. P.

Simulation in Healthcare, 12(3):148-156, June 2017 (article)

hi

[BibTex]

[BibTex]


no image
Probabilistic Prioritization of Movement Primitives

Paraschos, A., Lioutikov, R., Peters, J., Neumann, G.

Proceedings of the International Conference on Intelligent Robot Systems, and IEEE Robotics and Automation Letters (RA-L), PP(99), June 2017 (article) In press

ei

link (url) [BibTex]

link (url) [BibTex]


Thumb xl apollo system2 croped
Model-Based Policy Search for Automatic Tuning of Multivariate PID Controllers

Doerr, A., Nguyen-Tuong, D., Marco, A., Schaal, S., Trimpe, S.

In Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 5295-5301, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

am ics

PDF arXiv DOI Project Page [BibTex]

PDF arXiv DOI Project Page [BibTex]


Thumb xl learning ct block diagram v2
Learning Feedback Terms for Reactive Planning and Control

Rai, A., Sutanto, G., Schaal, S., Meier, F.

Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (conference)

am

pdf video [BibTex]

pdf video [BibTex]


Thumb xl this one
Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization

Marco, A., Berkenkamp, F., Hennig, P., Schoellig, A. P., Krause, A., Schaal, S., Trimpe, S.

In Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 1557-1563, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

am ics pn

PDF arXiv ICRA 2017 Spotlight presentation Virtual vs. Real - Video explanation DOI Project Page [BibTex]

PDF arXiv ICRA 2017 Spotlight presentation Virtual vs. Real - Video explanation DOI Project Page [BibTex]


Thumb xl screen shot 2017 06 14 at 2.38.22 pm
Scalable Pneumatic and Tendon Driven Robotic Joint Inspired by Jumping Spiders

Spröwitz, A., Göttler, C., Sinha, A., Caer, C., Öztekin, M. U., Petersen, K., Sitti, M.

In Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 64-70, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

dlg

Video link (url) DOI Project Page [BibTex]

Video link (url) DOI Project Page [BibTex]


Thumb xl publications toc
Design and actuation of a magnetic millirobot under a constant unidirectional magnetic field

Erin, O., Giltinan, J., Tsai, L., Sitti, M.

In Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 3404-3410, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

Abstract
Magnetic untethered millirobots, which are actuated and controlled by remote magnetic fields, have been proposed for medical applications due to their ability to safely pass through tissues at long ranges. For example, magnetic resonance imaging (MRI) systems with a 3-7 T constant unidirectional magnetic field and 3D gradient coils have been used to actuate magnetic robots. Such magnetically constrained systems place limits on the degrees of freedom that can be actuated for untethered devices. This paper presents a design and actuation methodology for a magnetic millirobot that exhibits both position and orientation control in 2D under a magnetic field, dominated by a constant unidirectional magnetic field as found in MRI systems. Placing a spherical permanent magnet, which is free to rotate inside the millirobot and located away from the center of mass, allows the generation of net forces and torques with applied 3D magnetic field gradients. We model this system in a 3D planar case and experimentally demonstrate open-loop control of both position and orientation by the applied 2D field gradients. The actuation performance is characterized across the most important design variables, and we experimentally demonstrate that the proposed approach is feasible.

pi

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates

Gu*, S., Holly*, E., Lillicrap, T., Levine, S.

Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017, *equal contribution (conference)

ei

Arxiv [BibTex]

Arxiv [BibTex]


Thumb xl publications toc
Magnetically actuated soft capsule endoscope for fine-needle aspiration biopsy

Son, D., Dogan, M. D., Sitti, M.

In Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 1132-1139, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

Abstract
This paper presents a magnetically actuated soft capsule endoscope for fine-needle aspiration biopsy (B-MASCE) in the upper gastrointestinal tract. A thin and hollow needle is attached to the capsule, which can penetrate deeply into tissues to obtain subsurface biopsy sample. The design utilizes a soft elastomer body as a compliant mechanism to guide the needle. An internal permanent magnet provides a means for both actuation and tracking. The capsule is designed to roll towards its target and then deploy the biopsy needle in a precise location selected as the target area. B-MASCE is controlled by multiple custom-designed electromagnets while its position and orientation are tracked by a magnetic sensor array. In in vitro trials, B-MASCE demonstrated rolling locomotion and biopsy of a swine tissue model positioned inside an anatomical human stomach model. It was confirmed after the experiment that a tissue sample was retained inside the needle.

pi

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
Proton 2: Increasing the Sensitivity and Portability of a Visuo-haptic Surface Interaction Recorder

Burka, A., Rajvanshi, A., Allen, S., Kuchenbecker, K. J.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages: 439-445, May 2017 (inproceedings)

Abstract
The Portable Robotic Optical/Tactile ObservatioN PACKage (PROTONPACK, or Proton for short) is a new handheld visuo-haptic sensing system that records surface interactions. We previously demonstrated system calibration and a classification task using external motion tracking. This paper details improvements in surface classification performance and removal of the dependence on external motion tracking, necessary before embarking on our goal of gathering a vast surface interaction dataset. Two experiments were performed to refine data collection parameters. After adjusting the placement and filtering of the Proton's high-bandwidth accelerometers, we recorded interactions between two differently-sized steel tooling ball end-effectors (diameter 6.35 and 9.525 mm) and five surfaces. Using features based on normal force, tangential force, end-effector speed, and contact vibration, we trained multi-class SVMs to classify the surfaces using 50 ms chunks of data from each end-effector. Classification accuracies of 84.5% and 91.5% respectively were achieved on unseen test data, an improvement over prior results. In parallel, we pursued on-board motion tracking, using the Proton's camera and fiducial markers. Motion tracks from the external and onboard trackers agree within 2 mm and 0.01 rad RMS, and the accuracy decreases only slightly to 87.7% when using onboard tracking for the 9.525 mm end-effector. These experiments indicate that the Proton 2 is ready for portable data collection.

hi

DOI [BibTex]

DOI [BibTex]


Thumb xl image toc
Propulsion and Chemotaxis in Bacteria-Driven Microswimmers

Zhuang, J., Park, B., Sitti, M.

Advanced Science, May 2017 (article)

Abstract
Despite the large body of experimental work recently on biohybrid microsystems, few studies have focused on theoretical modeling of such systems, which is essential to understand their underlying functioning mechanisms and hence design them optimally for a given application task. Therefore, this study focuses on developing a mathematical model to describe the 3D motion and chemotaxis of a type of widely studied biohybrid microswimmer, where spherical microbeads are driven by multiple attached bacteria. The model is developed based on the biophysical observations of the experimental system and is validated by comparing the model simulation with experimental 3D swimming trajectories and other motility characteristics, including mean squared displacement, speed, diffusivity, and turn angle. The chemotaxis modeling results of the microswimmers also agree well with the experiments, where a collective chemotactic behavior among multiple bacteria is observed. The simulation result implies that such collective chemotaxis behavior is due to a synchronized signaling pathway across the bacteria attached to the same microswimmer. Furthermore, the dependencies of the motility and chemotaxis of the microswimmers on certain system parameters, such as the chemoattractant concentration gradient, swimmer body size, and number of attached bacteria, toward an optimized design of such biohybrid system are studied. The optimized microswimmers would be used in targeted cargo, e.g., drug, imaging agent, gene, and RNA, transport and delivery inside the stagnant or low-velocity fluids of the human body as one of their potential biomedical applications.

pi

DOI Project Page Project Page [BibTex]


Thumb xl image toc
Dynamic and programmable self-assembly of micro-rafts at the air-water interface

Wang, W., Giltinan, J., Zakharchenko, S., Sitti, M.

Science Advances, 3(5):e1602522, American Association for the Advancement of Science, May 2017 (article)

Abstract
Dynamic self-assembled material systems constantly consume energy to maintain their spatiotemporal structures and functions. Programmable self-assembly translates information from individual parts to the collective whole. Combining dynamic and programmable self-assembly in a single platform opens up the possibilities to investigate both types of self-assembly simultaneously and to explore their synergy. This task is challenging because of the difficulty in finding suitable interactions that are both dissipative and programmable. We present a dynamic and programmable self-assembling material system consisting of spinning at the air-water interface circular magnetic micro-rafts of radius 50 μm and with cosinusoidal edge-height profiles. The cosinusoidal edge-height profiles not only create a net dissipative capillary repulsion that is sustained by continuous torque input but also enable directional assembly of micro-rafts. We uncover the layered arrangement of micro-rafts in the patterns formed by dynamic self-assembly and offer mechanistic insights through a physical model and geometric analysis. Furthermore, we demonstrate programmable self-assembly and show that a 4-fold rotational symmetry encoded in individual micro-rafts translates into 90° bending angles and square-based tiling in the assembled structures of micro-rafts. We anticipate that our dynamic and programmable material system will serve as a model system for studying nonequilibrium dynamics and statistical mechanics in the future

pi

DOI Project Page Project Page Project Page [BibTex]

DOI Project Page Project Page Project Page [BibTex]


Thumb xl emthy 01
Presentation of functional groups on self-assembled supramolecular peptide nanofibers mimicking glycosaminoglycans for directed mesenchymal stem cell differentiation

Yasa, O., Uysal, O., Ekiz, M. S., Guler, M. O., Tekinay, A. B.

J. Mater. Chem. B, 5, pages: 4890-4900, The Royal Society of Chemistry, May 2017 (article)

Abstract
Organizational complexity and functional diversity of the extracellular matrix regulate cellular behaviors. The extracellular matrix is composed of various proteins in the form of proteoglycans{,} glycoproteins{,} and nanofibers whose types and combinations change depending on the tissue type. Proteoglycans{,} which are proteins that are covalently attached to glycosaminoglycans{,} contribute to the complexity of the microenvironment of the cells. The sulfation degree of the glycosaminoglycans is an important and distinct feature at specific developmental stages and tissue types. Peptide amphiphile nanofibers can mimic natural glycosaminoglycans and/or proteoglycans{,} and they form a synthetic nanofibrous microenvironment where cells can proliferate and differentiate towards different lineages. In this study{,} peptide nanofibers were used to provide varying degrees of sulfonation mimicking the natural glycosaminoglycans by forming a microenvironment for the survival and differentiation of stem cells. The effects of glucose{,} carboxylate{,} and sulfonate groups on the peptide nanofibers were investigated by considering the changes in the differentiation profiles of rat mesenchymal stem cells in the absence of any specific differentiation inducers in the culture medium. The results showed that a higher sulfonate-to-glucose ratio is associated with adipogenic differentiation and a higher carboxylate-to-glucose ratio is associated with osteochondrogenic differentiation of the rat mesenchymal stem cells. Overall{,} these results demonstrate that supramolecular peptide nanosystems can be used to understand the fine-tunings of the extracellular matrix such as sulfation profile on specific cell types.

pi

link (url) DOI [BibTex]

link (url) DOI [BibTex]


Thumb xl drotlef et al 2017 advanced materials
Bioinspired Composite Microfibers for Skin Adhesion and Signal Amplification of Wearable Sensors

Drotlef, D., Amjadi, M., Yunusa, M., Sitti, M.

Advanced Materials, May 2017, Back Cover (article)

Abstract
A facile approach is proposed for superior conformation and adhesion of wearable sensors to dry and wet skin. Bioinspired skin-adhesive films are composed of elastomeric microfibers decorated with conformal and mushroom-shaped vinylsiloxane tips. Strong skin adhesion is achieved by crosslinking the viscous vinylsiloxane tips directly on the skin surface. Furthermore, composite microfibrillar adhesive films possess a high adhesion strength of 18 kPa due to the excellent shape adaptation of the vinylsiloxane tips to the multiscale roughness of the skin. As a utility of the skin-adhesive films in wearable-device applications, they are integrated with wearable strain sensors for respiratory and heart-rate monitoring. The signal-to-noise ratio of the strain sensor is significantly improved to 59.7 because of the considerable signal amplification of microfibrillar skin-adhesive films.

pi

DOI [BibTex]


Thumb xl mostaghaci et al 2017 advanced science
Bioadhesive Bacterial Microswimmers for Targeted Drug Delivery in the Urinary and Gastrointestinal Tracts

Mostaghaci, B., Yasa, O., Zhuang, J., Sitti, M.

Advanced Science, May 2017 (article)

Abstract
Bacteria-driven biohybrid microswimmers (bacteriabots), which integrate motile bacterial cells and functional synthetic cargo parts (e.g., microparticles encapsulating drug), are recently studied for targeted drug delivery. However, adhesion of such bacteriabots to the tissues on the site of a disease (which can increase the drug delivery efficiency) is not studied yet. Here, this paper proposes an approach to attach bacteriabots to certain types of epithelial cells (expressing mannose on the membrane), based on the affinity between lectin molecules on the tip of bacterial type I pili and mannose molecules on the epithelial cells. It is shown that the bacteria can anchor their cargo particles to mannose-functionalized surfaces and mannose-expressing cells (ATCC HTB-9) using the lectin–mannose bond. The attachment mechanism is confirmed by comparing the adhesion of bacteriabots fabricated from bacterial strains with or without type I pili to mannose-covered surfaces and cells. The proposed bioadhesive motile system can be further improved by expressing more specific adhesion moieties on the membrane of the bacteria.

pi

DOI Project Page [BibTex]


Thumb xl image toc
Six Degree-of-Freedom Localization of Endoscopic Capsule Robots using Recurrent Neural Networks embedded into a Convolutional Neural Network

Turan, M., Abdullah, A., Jamiruddin, R., Araujo, H., Konukoglu, E., Sitti, M.

arXiv preprint arXiv:1705.06196, May 2017 (article)

Abstract
Since its development, ingestible wireless endoscopy is considered to be a painless diagnostic method to detect a number of diseases inside GI tract. Medical related engineering companies have made significant improvements in this technology in last decade; however, some major limitations still residue. Localization of the next generation steerable endoscopic capsule robot in six degreeof-freedom (DoF) and active motion control are some of these limitations. The significance of localization capability concerns with the doctors correct diagnosis of the disease area. This paper presents a very robust 6-DoF localization method based on supervised training of an architecture consisting of recurrent networks (RNN) embedded into a convolutional neural network (CNN) to make use of both just-in-moment information obtained by CNN and correlative information across frames obtained by RNN. To our knowledge, our idea of embedding RNNs into a CNN architecture is for the first time proposed in literature. The experimental results show that the proposed RNN-in-CNN architecture performs very well for endoscopic capsule robot localization in cases vignetting, reflection distortions, noise, sudden camera movements and lack of distinguishable features.

pi

DOI Project Page [BibTex]


Thumb xl publications toc
Controllable load sharing for soft adhesive interfaces on three-dimensional surfaces

Song, S., Drotlef, D., Majidi, C., Sitti, M.

Proceedings of the National Academy of Sciences, pages: 201620344, National Acad Sciences, May 2017 (article)

Abstract
For adhering to three-dimensional (3D) surfaces or objects, current adhesion systems are limited by a fundamental trade-off between 3D surface conformability and high adhesion strength. This limitation arises from the need for a soft, mechanically compliant interface, which enables conformability to nonflat and irregularly shaped surfaces but significantly reduces the interfacial fracture strength. In this work, we overcome this trade-off with an adhesion-based soft-gripping system that exhibits enhanced fracture strength without sacrificing conformability to nonplanar 3D surfaces. Composed of a gecko-inspired elastomeric microfibrillar adhesive membrane supported by a pressure-controlled deformable gripper body, the proposed soft-gripping system controls the bonding strength by changing its internal pressure and exploiting the mechanics of interfacial equal load sharing. The soft adhesion system can use up to ∼26% of the maximum adhesion of the fibrillar membrane, which is 14× higher than the adhering membrane without load sharing. Our proposed load-sharing method suggests a paradigm for soft adhesion-based gripping and transfer-printing systems that achieves area scaling similar to that of a natural gecko footpad.

pi

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Thumb xl image toc
A Non-Rigid Map Fusion-Based RGB-Depth SLAM Method for Endoscopic Capsule Robots

Turan, M., Almalioglu, Y., Araujo, H., Konukoglu, E., Sitti, M.

arXiv preprint arXiv:1705.05444, May 2017 (article)

Abstract
In the gastrointestinal (GI) tract endoscopy field, ingestible wireless capsule endoscopy is considered as a minimally invasive novel diagnostic technology to inspect the entire GI tract and to diagnose various diseases and pathologies. Since the development of this technology, medical device companies and many groups have made significant progress to turn such passive capsule endoscopes into robotic active capsule endoscopes to achieve almost all functions of current active flexible endoscopes. However, the use of robotic capsule endoscopy still has some challenges. One such challenge is the precise localization of such active devices in 3D world, which is essential for a precise three-dimensional (3D) mapping of the inner organ. A reliable 3D map of the explored inner organ could assist the doctors to make more intuitive and correct diagnosis. In this paper, we propose to our knowledge for the first time in literature a visual simultaneous localization and mapping (SLAM) method specifically developed for endoscopic capsule robots. The proposed RGB-Depth SLAM method is capable of capturing comprehensive dense globally consistent surfel-based maps of the inner organs explored by an endoscopic capsule robot in real time. This is achieved by using dense frame-to-model camera tracking and windowed surfelbased fusion coupled with frequent model refinement through non-rigid surface deformations.

pi

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]