Publications
2023
- In preparationOperating Autonomous Seaweed Farms by Going With The Flow of Ocean CurrentsMarius Wiggert, Manan Doshi, Pierre FJ Lermusiaux, and Claire J TomlinIn preparation for Science Robotics, 2023
- In preparationHedging against Uncertainty with Data-Assimilative Path Planning and POMPDsMarius Wiggert*, Manan Doshi*, Pierre FJ Lermusiaux, and Claire J TomlinIn preparation for IEEE Robotics and Automation Letters (RA-L), 2023
- PhD ThesisTowards Operating Underactuated Robotic Systems by Going With the FlowMarius WiggertEECS Department, University of California, Berkeley, 2023
Over the centuries, humanity has created ever more ingenious systems to traverse the oceans and skies of our planet. Modern ships and planes operate with powerful engines that require substantial amounts of fuel, leading to high operating costs. However, this approach becomes impractical for applications that require extended periods of autonomous operation without the possibility of refueling. This dissertation starts with the idea of operating systems by going with the flow: harnessing the wind and ocean currents by letting the system drift in favorable directions and strategically using a low-power engine to change flows when this is beneficial. As the power to counteract drag forces scales cubically with the relative velocity of the system, this new paradigm reduces the power required for operation by 2-3 orders of magnitude, thereby significantly reducing the system and operating costs. This could enable a host of novel applications that require low-cost and long-term operations, such as active environmental monitoring of the oceans and atmosphere or floating solar platforms. The primary case study used throughout this work is autonomous seaweed farms that roam the oceans while rapidly growing biomass for biofuel, bioplastic, or to sink it for carbon removal. In this dissertation, we systematically develop control techniques to tackle the four key challenges of operating by going with the flow: First, the system is severely underactuated with its own propulsion often being less than 1/10th of the magnitude of the surrounding flows. Second, to make strategic control decisions when to change flows, only coarse, deterministic forecasts are available. Third, the forecasts have a limited time horizon of 5-10 days, but realistic control objectives extend over weeks to months. Lastly, the forecast error defined as the difference between the forecasted and the true flows often exceeds the propulsion capabilities of the system, hence robust control is infeasible. We start by introducing techniques for continuous-time optimal control when the complex flows are known. We use dynamic programming for the objectives of navigation and maximizing seaweed growth. Next, we turn towards the challenge of operating with imperfect and short-term forecasts. Our insight is that the value functions obtained by the previously developed optimal control methods can be used as closed-loop control policies, which are equivalent to replanning on the forecast at every step. Through extensive simulation studies in realistic ocean conditions, we demonstrate that such frequent replanning allows for reliable operation despite significant forecast errors. To enable reasoning beyond the forecast horizon, we derive a discounted optimal control formulation and demonstrate how the value function can be extended by estimating the cost-to-go using historical flow averages. In the last part of this dissertation, we focus on how to handle constraints in these challenging environments. For that, we integrate time-varying obstacles into our value function and show empirically that this almost eliminates the risk of stranding. Moreover, we develop a hierarchical control approach to operate a fleet of underactuated autonomous systems while avoiding collisions and ensuring connectivity across the fleet. At the end of this dissertation, we summarize our techniques for operating by going with the flow which could enable a host of new applications of low-power autonomous systems in the oceans and skies. We also discuss promising ongoing and future research directions towards further improving the performance of underactuated robotic systems operating in flows.
- ICRAMaximizing seaweed growth on autonomous farms: A dynamic programming approach for underactuated systems operating on uncertain ocean currentsMatthias Killer*, Marius Wiggert*, Hanna Krasowski, Manan Doshi, Pierre FJ Lermusiaux, and Claire J TomlinSubmitted to IEEE International Conference on Robotics and Automation (ICRA), 2024; arXiv preprint arXiv:2307.01916, 2023
Seaweed biomass presents a substantial opportunity for climate mitigation, yet to realize its potential, farming must be expanded to the expansive open oceans. However, in the open ocean neither anchored farming nor floating farms operating with powerful engines are economically viable. Recent studies have shown that vessels can navigate with low-power engines by going with the flow, utilizing minimal propulsion to strategically leverage beneficial ocean currents. In this work, we focus on low-power autonomous seaweed farms and design controllers that maximize seaweed growth by taking advantage of ocean currents. We first introduce a Dynamic Programming (DP) formulation to solve for the growth-optimal value function when the true currents are known. However, in reality only short-term imperfect forecasts with increasing uncertainty are available. Hence, we present three additional extensions. Firstly, we use frequent replanning to mitigate forecast errors. For that we compute the value function daily as new forecasts arrive, which also provides a feedback policy that is equivalent to replanning on the forecast at every time step. Second, to optimize for long-term growth, we extend the value function beyond the forecast horizon by estimating the expected future growth based on seasonal average currents. Lastly, we introduce a discounted finite-time DP formulation to account for the increasing uncertainty in future ocean current estimates. We empirically evaluate our approach with 30-day simulations of farms in realistic ocean conditions. Our method achieves 95.8% of the best possible growth using only 5-day forecasts. This confirms the feasibility of using low-power propulsion to operate autonomous farms in real-world conditions.
- T-CSTSafe Connectivity Maintenance in Underactuated Multi-Agent Networks for Dynamic Oceanic EnvironmentsNicolas Hoischen*, Marius Wiggert*, and Claire J TomlinSubmitted to IEEE Journal Transactions on Control Systems Technology; arXiv preprint arXiv:2307.01927, 2023
Autonomous Multi-Agent Systems are increasingly being deployed in environments where winds and ocean currents can exert a significant influence on their dynamics. Recent work has developed powerful control policies for single agents that can leverage flows to achieve their objectives in dynamic environments. However, in the context of multi-agent systems, these flows can cause agents to collide or drift apart and lose direct inter-agent communications, especially when agents have low propulsion capabilities. To address these challenges, we propose a Hierarchical Multi-Agent Control approach that allows arbitrary single agent performance policies that are unaware of other agents to be used in multi-agent systems, while ensuring safe operation. We first develop a safety controller solely dedicated to avoiding collisions and maintaining inter-agent communication. Subsequently, we design a lowinterference safe interaction (LISIC) policy that trades-off the performance policy and the safety controller to ensure safe and optimal operation. Specifically, when the agents are at an appropriate distance, LISIC prioritizes the performance policy, while smoothly increasing the safety controller when necessary. We prove that under mild assumptions on the flows experienced by the agents our approach can guarantee safety. Additionally, we demonstrate the effectiveness of our method in realistic settings through an extensive empirical analysis with underactuated Autonomous Surface Vehicles (ASV) operating in dynamical ocean currents where the assumptions do not always hold.
- CDCStranding Risk for Underactuated Vessels in Complex Ocean Currents: Analysis and ControllersAndreas Doering*, Marius Wiggert*, Hanna Krasowski, Manan Doshi, Pierre FJ Lermusiaux, and Claire J Tomlin2023 IEEE 62th Conference on Decision and Control (CDC); arXiv preprint arXiv:2307.01917, 2023
Low-propulsion vessels can take advantage of powerful ocean currents to navigate towards a destination. Recent results demonstrated that vessels can reach their destination with high probability despite forecast errors. However, these results do not consider the critical aspect of safety of such vessels: because of their low propulsion which is much smaller than the magnitude of currents, they might end up in currents that inevitably push them into unsafe areas such as shallow areas, garbage patches, and shipping lanes. In this work, we first investigate the risk of stranding for free-floating vessels in the Northeast Pacific. We find that at least 5.04% would strand within 90 days. Next, we encode the unsafe sets as hard constraints into Hamilton-Jacobi Multi-Time Reachability (HJ-MTR) to synthesize a feedback policy that is equivalent to re-planning at each time step at low computational cost. While applying this policy closed-loop guarantees safe operation when the currents are known, in realistic situations only imperfect forecasts are available. We demonstrate the safety of our approach in such realistic situations empirically with large-scale simulations of a vessel navigating in high-risk regions in the Northeast Pacific. We find that applying our policy closed-loop with daily re-planning on new forecasts can ensure safety with high probability even under forecast errors that exceed the maximal propulsion. Our method significantly improves safety over the baselines and still achieves a timely arrival of the vessel at the destination.
2022
- IJRRInducing structure in reward learning by learning featuresAndreea Bobu, Marius Wiggert, Claire Tomlin, and Anca D DraganThe International Journal of Robotics Research, 2022
Reward learning enables robots to learn adaptable behaviors from human input. Traditional methods model the reward as a linear function of hand-crafted features, but that requires specifying all the relevant features a priori, which is impossible for real-world tasks. To get around this issue, recent deep Inverse Reinforcement Learning (IRL) methods learn rewards directly from the raw state but this is challenging because the robot has to implicitly learn the features that are important and how to combine them, simultaneously. Instead, we propose a divide-and-conquer approach: focus human input specifically on learning the features separately, and only then learn how to combine them into a reward. We introduce a novel type of human input for teaching features and an algorithm that utilizes it to learn complex features from the raw state space. The robot can then learn how to combine them into a reward using demonstrations, corrections, or other reward learning frameworks. We demonstrate our method in settings where all features have to be learned from scratch, as well as where some of the features are known. By first focusing human input specifically on the feature(s), our method decreases sample complexity and improves generalization of the learned reward over a deep IRL baseline. We show this in experiments with a physical 7-DoF robot manipulator, and in a user study conducted in a simulated environment.
- CDCNavigating underactuated agents by hitchhiking forecast flowsMarius Wiggert, Manan Doshi, Pierre FJ Lermusiaux, and Claire J TomlinIn 2022 IEEE 61st Conference on Decision and Control (CDC), 2022
In dynamic flow fields such as winds and ocean currents an agent can navigate by going with the flow, only using minimal propulsion to nudge itself into beneficial flows. This navigation paradigm of hitchhiking flows is highly energy-efficient. However, reliable navigation in this setting remains challenging as typically only forecasts are available which differ significantly from the true currents and the forecast error can be larger than can be handled by the actuation of the agent. In this paper, we propose a novel control method for reliable navigation of underactuated agents hitchhiking flows based on imperfect forecasts. In the spirit of Model Predictive Control our method allows for time-optimal replanning at every time step with only one computation per forecast. Using the recent Multi-Time Hamilton-Jacobi Reachability formulation we obtain a value function which is then used for closed-loop control. We evaluate the reliability of our method empirically over a large set of multi-day start-target missions in the ocean currents of the Gulf of Mexico with realistic forecast errors. Our method outperforms the baselines significantly, achieving high reliability, measured as the success rate of navigating from start to target, at low computational cost.
- CDCHamilton-jacobi multi-time reachabilityManan Doshi, Manmeet Bhabra, Marius Wiggert, Claire J Tomlin, and Pierre FJ LermusiauxIn 2022 IEEE 61st Conference on Decision and Control (CDC), 2022
For the analysis of dynamical systems, it is fundamental to determine all states that can be reached at any given time. In this work, we obtain and apply new governing equations for reachability analysis over multiple start and terminal times all at once, and for systems operating in time-varying environments with dynamic obstacles and any other relevant dynamic fields. The theory and schemes are developed for both backward and forward reachable tubes with time-varying target and start sets. The resulting value functions elegantly capture not only the reachable tubes but also time-to-reach and time-to-leave maps as well as start time vs. duration plots and other useful secondary quantities for optimal control. We discuss the numerical schemes and computational efficiency. We first verify our results in an environment with a moving target and obstacle where reachability tubes can be analytically computed. We then consider the Dubin’s car problem extended with a moving target and obstacle. Finally, we showcase our multi-time reachability in a non-hydrostatic bottom gravity current system. Results highlight the novel capabilities of exact multi-time reachability in dynamic environments.
2021
- HRIFeature expansive reward learning: Rethinking human inputAndreea Bobu*, Marius Wiggert*, Claire Tomlin, and Anca D DraganIn Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 2021
When a person is not satisfied with how a robot performs a task, they can intervene to correct it. Reward learning methods enable the robot to adapt its reward function online based on such human input, but they rely on handcrafted features. When the correction cannot be explained by these features, recent work in deep Inverse Reinforcement Learning (IRL) suggests that the robot could ask for task demonstrations and recover a reward defined over the raw state space. Our insight is that rather than implicitly learning about the missing feature(s) from demonstrations, the robot should instead ask for data that explicitly teaches it about what it is missing. We introduce a new type of human input in which the person guides the robot from states where the feature being taught is highly expressed to states where it is not. We propose an algorithm for learning the feature from the raw state space and integrating it into the reward function. By focusing the human input on the missing feature, our method decreases sample complexity and improves generalization of the learned reward over the above deep IRL baseline. We show this in experiments with a physical 7DOF robot manipulator, as well as in a user study conducted in a simulated environment.
- ACCIdentification of Cancer Cell Population Dynamics Leveraging the Effect of Pre-Treatment for Drug Schedule DesignMarius Wiggert, Megan Turnidge, Zoe Cohen, Ellen M Langer, Rosalie C Sears, Margaret P Chapman, and Claire J TomlinIn 2021 American Control Conference (ACC), 2021
Sequences of different drugs have shown potential to improve treatment strategies for cancer. Typical switched system approaches model the population dynamics of each drug independently, not rigorously considering the effects of pretreatment or drug-drug interactions. In this paper, a general model family incorporating pre-treatment effects and biological domain knowledge is proposed, and a model from this family is identified by using a novel experimental data set of two-drug sequences. Leveraging the data, a simulator for the cell population dynamics under sequences of up to nine drugs is developed and used to empirically evaluate the performance of a set of closed-loop drug scheduling controllers. We used the controllers to identifying promising drug schedules in silico and evaluated them in vitro. The experiments validated the effectiveness of the identified schedules in reducing the number of living cells to less than 10% of the initial. While only treating with certain toxic drugs achieves similar effectiveness, the schedules use toxic drugs for significantly shorter times which likely reduces toxicity to non-cancer cells.
2019
- CASERapid-molt: A meso-scale, open-source, low-cost testbed for robot assisted precision irrigation and deliveryMarius Wiggert, Leela Amladi, Ron Berenstein, Stefano Carpin, Joshua Viers, Stavros Vougioukas, and Ken GoldbergIn 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE), 2019
To study the automation of plant-level precision irrigation, specifically learning-based irrigation controllers, we present a modular, open-source testbed that enables real-time, fine-grained data collection and irrigation actuation. RAPID-MOLT costs USD $600 and has floor space of 0.37m 2 . The functionality of the platform is evaluated by measuring the correlation between plant growth (Leaf Area Index) and water stress (Crop Water Stress Index) with irrigation volume. In line with biological studies, the observed plant growth is positively correlated with irrigation volume while water stress is negatively correlated. Construction directions, experimental data, CAD models, and related software are available at github.com/BerkeleyAutomation/RAPID-MOLT.