Match of the Day presenter Gary Lineker has sealed a new agreement with the BBC just hours after his exit was confirmed.

PGMOL chief Howard Webb has responded after referee David Coote was suspended for comments he appeared to make in a video.

Emma Raducanu struck a deal to return to one of her favourite tournaments.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

ICRA@40: 23–26 September 2024, ROTTERDAM, NETHERLANDS

IROS 2024: 14–18 October 2024, ABU DHABI, UAE

ICSR 2024: 23–26 October 2024, ODENSE, DENMARK

Cybathlon 2024: 25–27 October 2024, ZURICH

Enjoy today’s videos!

The National Science Foundation Human AugmentatioN via Dexterity Engineering Research Center (HAND ERC) was announced in August 2024. Funded for up to 10 years and $52 million, the HAND ERC is led by Northwestern University, with core members Texas A&M, Florida A&M, Carnegie Mellon, and MIT, and support from Wisconsin-Madison, Syracuse, and an innovation ecosystem consisting of companies, national labs, and civic and advocacy organizations. HAND will develop versatile, easy-to-use dexterous robot end effectors (hands).

[ HAND ]

The Environmental Robotics Lab at ETH Zurich, in partnership with Wilderness International (and some help from DJI and Audi), is using drones to sample DNA from the tops of trees in the Peruvian rainforest. Somehow, the treetops are where 60 to 90 percent of biodiversity is found, and these drones can help researchers determine what the heck is going on up there.

[ ERL ]

Thanks, Steffen!

1X introduces NEO Beta, “the pre-production build of our home humanoid.”

“Our priority is safety,” said Bernt Børnich, CEO at 1X. “Safety is the cornerstone that allows us to confidently introduce NEO Beta into homes, where it will gather essential feedback and demonstrate its capabilities in real-world settings. This year, we are deploying a limited number of NEO units in selected homes for research and development purposes. Doing so means we are taking another step toward achieving our mission.”

[ 1X ]

We love MangDang’s fun and affordable approach to robotics with Mini Pupper. The next generation of the little legged robot has just launched on Kickstarter, featuring new and updated robots that make it easy to explore embodied AI.

The Kickstarter is already fully funded after just a day or two, but there are still plenty of robots up for grabs.

[ Kickstarter ]

Quadrupeds in space can use their legs to reorient themselves. Or, if you throw one off a roof, it can learn to land on its feet.

To be presented at CoRL 2024.

[ ARL ]

HEBI Robotics, which apparently was once headquartered inside a Pittsburgh public bus, has imbued a table with actuators and a mind of its own.

[ HEBI Robotics ]

Carcinization is a concept in evolutionary biology where a crustacean that isn’t a crab eventually becomes a crab. So why not do the same thing with robots? Crab robots solve all problems!

[ KAIST ]

Waymo is smart, but also humans are really, really dumb sometimes.

[ Waymo ]

The Robotics Department of the University of Michigan created an interactive community art project. The group that led the creation believed that while roboticists typically take on critical and impactful problems in transportation, medicine, mobility, logistics, and manufacturing, there are many opportunities to find play and amusement. The final piece is a grid of art boxes, produced by different members of our robotics community, which offer an eight-inch-square view into their own work with robotics.

[ Michigan Robotics ]

I appreciate that UBTECH’s humanoid is doing an actual job, but why would you use a humanoid for this?

[ UBTECH ]

I’m sure most actuators go through some form of life-cycle testing. But if you really want to test an electric motor, put it into a BattleBot and see what happens.

[ Hardcore Robotics ]

Yes, but have you tried fighting a BattleBot?

[ AgileX ]

In this video, we present collaboration aerial grasping and transportation using multiple quadrotors with cable-suspended payloads. Grasping using a suspended gripper requires accurate tracking of the electromagnet to ensure a successful grasp while switching between different slack and taut modes. In this work, we grasp the payload using a hybrid control approach that switches between a quadrotor position control and a payload position control based on cable slackness. Finally, we use two quadrotors with suspended electromagnet systems to collaboratively grasp and pick up a larger payload for transportation.

[ Hybrid Robotics ]

I had not realized that the floretizing of broccoli was so violent.

[ Oxipital ]

While the RoboCup was held over a month ago, we still wanted to make a small summary of our results, the most memorable moments, and of course an homage to everyone who is involved with the B-Human team: the team members, the sponsors, and the fans at home. Thank you so much for making B-Human the team it is!

[ B-Human ]

This is a sponsored article brought to you by Khalifa University of Science and Technology.

Abu Dhabi-based Khalifa University of Science and Technology in the United Arab Emirates (UAE) will be hosting the 36th edition of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024) to highlight the Middle East and North Africa (MENA) region’s rapidly advancing capabilities in the robotics and intelligent transport systems.

aspect_ratio

Themed “Robotics for Sustainable Development,” the IROS 2024 will be held from 14-18 October 2024 at the Abu Dhabi National Exhibition Center (ADNEC) in the UAE’s capital city. It will offer a platform for universities and research institutions to display their research and innovation activities and initiatives in robotics, gathering researchers, academics, leading corporate majors, and industry professionals from around the globe.

A total of 13 forums, nine global-level competitions and challenges covering various aspects of robotics and AI, an IROS Expo, as well as an exclusive Career Fair will also be part of IROS 2024. The challenges and competitions will focus on physical or athletic intelligence of robots, remote robot navigation, robot manipulation, underwater robotics, as well as perception and sensing.

Delegates for the event will represent sectors including manufacturing, healthcare, logistics, agriculture, defense, security, and mining sectors with 60 percent of the talent pool having over six years of experience in robotics. A major component of the conference will be the poster sessions, keynotes, panel discussions by researchers and scientists, and networking events.

Khalifa University will be hosting IROS 2024 to highlight the Middle East and North Africa (MENA) region’s rapidly advancing capabilities in the robotics and intelligent transport systems.Khalifa University

Abu Dhabi ranks first on the world’s safest cities list in 2024, according to online database Numbeo, out of 329 global cities in the 2024 standings, holding the title for eight consecutive years since 2017, reflecting the emirate’s ongoing efforts to ensure a good quality of life for citizens and residents.

With a multicultural community, Abu Dhabi is home to people from more than 200 nationalities and draws a large number of tourists to some of the top art galleries in the city such as Louvre Abu Dhabi and the Guggenheim Abu Dhabi, as well as other destinations such as Ferrari World Abu Dhabi and Warner Bros. World Abu Dhabi.

The UAE and Abu Dhabi have increasingly become a center for creative skillsets, human capital and advanced technologies, attracting several international and regional events such as the global COP28 UAE climate summit, in which more than 160 countries participated.

Abu Dhabi city itself has hosted a number of association conventions such as the 34th International Nursing Research Congress and is set to host the UNCTAD World Investment Forum, the 13th World Trade Organization (WTO) Ministerial Conference (MC13), the 12th World Environment Education Congress in 2024, and the IUCN World Conservation Congress in 2025.

Khalifa University’s Center for Robotics and Autonomous Systems (KU-CARS) includes a vibrant multidisciplinary environment for conducting robotics and autonomous vehicle-related research and innovation.Khalifa University

Dr. Jorge Dias, IROS 2024 General Chair, said: “Khalifa University is delighted to bring the Intelligent Robots and Systems 2024 to Abu Dhabi in the UAE and highlight the innovations in line with the theme Robotics for Sustainable Development. As the region’s rapidly advancing capabilities in robotics and intelligent transport systems gain momentum, this event serves as a platform to incubate ideas, exchange knowledge, foster collaboration, and showcase our research and innovation activities. By hosting IROS 2024, Khalifa University aims to reaffirm the UAE’s status as a global innovation hub and destination for all industry stakeholders to collaborate on cutting-edge research and explore opportunities for growth within the UAE’s innovation ecosystem.”

“This event serves as a platform to incubate ideas, exchange knowledge, foster collaboration, and showcase our research and innovation activities” —Dr. Jorge Dias, IROS 2024 General Chair

Dr. Dias added: “The organizing committee of IROS 2024 has received over 4000 submissions representing 60 countries, with China leading with 1,029 papers, followed by the U.S. (777), Germany (302), and Japan (253), as well as the U.K. and South Korea (173 each). The UAE with a total of 68 papers comes atop the Arab region.”

Driving innovation at Khalifa University is the Center for Robotics and Autonomous Systems (KU-CARS) with around 50 researchers and state-of-the-art laboratory facilities, including a vibrant multidisciplinary environment for conducting robotics and autonomous vehicle-related research and innovation.

IROS 2024 is sponsored by IEEE Robotics and Automation Society, Abu Dhabi Convention and Exhibition Bureau, the Robotics Society of Japan (RSJ), the Society of Instrument and Control Engineers (SICE), the New Technology Foundation, and the IEEE Industrial Electronics Society (IES).

More information at https://iros2024-abudhabi.org/

The software used to control a robot is normally highly adapted to its specific physical set up. But now researchers have created a single general-purpose robotic control policy that can operate robotic arms, wheeled robots, quadrupeds, and even drones.

One of the biggest challenges when it comes to applying machine learning to robotics is the paucity of data. While computer vision and natural language processing can piggyback off the vast quantities of image and text data found on the Internet, collecting robot data is costly and time-consuming.

To get around this, there have been growing efforts to pool data collected by different groups on different kinds of robots, including the Open X-Embodiment and DROID datasets. The hope is that training on diverse robotics data will lead to “positive transfer,” which refers to when skills learned from training on one task help to boost performance on another.

The problem is that robots often have very different embodiments—a term used to describe their physical layout and suite of sensors and actuators—so the data they collect can vary significantly. For instance, a robotic arm might be static, have a complex arrangement of joints and fingers, and collect video from a camera on its wrist. In contrast, a quadruped robot is regularly on the move and relies on force feedback from its legs to maneuver. The kinds of tasks and actions these machines are trained to carry out are also diverse: The arm may pick and place objects, while the quadruped needs keen navigation.

That makes training a single AI model for robots on these large collections of data challenging, says Homer Walke, a Ph.D. student at the University of California, Berkeley. So far, most attempts have either focused on data from a narrower selection of similar robots or researchers have manually tweaked data to make observations from different robots more similar. But in research to be presented at the Conference on Robot Learning (CoRL) in Munich in November, they unveiled a new model called CrossFormer that can train on data from a diverse set of robots and control them just as well as specialized control policies.

“We want to be able to train on all of this data to get the most capable robot,” says Walke. “The main advance in this paper is working out what kind of architecture works the best for accommodating all these varying inputs and outputs.”

How to control diverse robots with the same AI model

The team used the same model architecture that powers large language model, known as a transformer. In many ways, the challenge the researchers were trying to solve is not dissimilar to that facing a chatbot, says Walke. In language modeling, the AI has to to pick out similar patterns in sentences with different lengths and word orders. Robot data can also be arranged in a sequence much like a written sentence, but depending on the particular embodiment, observations and actions vary in length and order too.

“Words might appear in different locations in a sentence, but they still mean the same thing,” says Walke. “In our task, an observation image might appear in different locations in the sequence, but it’s still fundamentally an image and we still want to treat it like an image.”

UC Berkeley/Carnegie Mellon University

Most machine learning approaches work through a sequence one element at a time, but transformers can process the entire stream of data at once. This allows them to analyze the relationship between different elements and makes them better at handling sequences that are not standardized, much like the diverse data found in large robotics datasets.

Walke and his colleagues aren’t the first to train transformers on large-scale robotics data. But previous approaches have either trained solely on data from robotic arms with broadly similar embodiments or manually converted input data to a common format to make it easier to process. In contrast, CrossFormer can process images from cameras positioned above a robot, at head height or on a robotic arms wrist, as well as joint position data from both quadrupeds and robotic arms, without any tweaks.

The result is a single control policy that can operate single robotic arms, pairs of robotic arms, quadrupeds, and wheeled robots on tasks as varied as picking and placing objects, cutting sushi, and obstacle avoidance. Crucially, it matched the performance of specialized models tailored for each robot and outperformed previous approaches trained on diverse robotic data. The team even tested whether the model could control an embodiment not included in the dataset—a small quadcopter. While they simplified things by making the drone fly at a fixed altitude, CrossFormer still outperformed the previous best method.

“That was definitely pretty cool,” says Ria Doshi, an undergraduate student at Berkeley. “I think that as we scale up our policy to be able to train on even larger sets of diverse data, it’ll become easier to see this kind of zero shot transfer onto robots that have been completely unseen in the training.”

The limitations of one AI model for all robots

The team admits there’s still work to do, however. The model is too big for any of the robots’ embedded chips and instead has to be run from a server. Even then, processing times are only just fast enough to support real-time operation, and Walke admits that could break down if they scale up the model. “When you pack so much data into a model it has to be very big and that means running it for real-time control becomes difficult.”

One potential workaround would be to use an approach called distillation, says Oier Mees, a postdoctoral research at Berkley and part of the CrossFormer team. This essentially involves training a smaller model to mimic the larger model, and if successful can result in similar performance for a much smaller computational budget.

But of more importance than the computing resource problem is that the team failed to see any positive transfer in their experiments, as CrossFormer simply matched previous performance rather than exceeding it. Walke thinks progress in computer vision and natural language processing suggests that training on more data could be the key.

Others say it might not be that simple. Jeannette Bohg, a professor of robotics at Stanford University, says the ability to train on such a diverse dataset is a significant contribution. But she wonders whether part of the reason why the researchers didn’t see positive transfer is their insistence on not aligning the input data. Previous research that trained on robots with similar observation and action data has shown evidence of such cross-overs. “By getting rid of this alignment, they may have also gotten rid of this significant positive transfer that we’ve seen in other work,” Bohg says.

It’s also not clear if the approach will boost performance on tasks specific to particular embodiments or robotic applications, says Ram Ramamoorthy, a robotics professor at Edinburgh University. The work is a promising step towards helping robots capture concepts common to most robots, like “avoid this obstacle,” he says. But it may be less useful for tackling control problems specific to a particular robot, such as how to knead dough or navigate a forest, which are often the hardest to solve.

Four decades after the first IEEE International Conference on Robotics and Automation (ICRA) in Atlanta, robotics is bigger than ever. Next week in Rotterdam is the IEEE ICRA@40 conference, “a celebration of 40 years of pioneering research and technological advancements in robotics and automation.” There’s an ICRA every year, of course. Arguably the largest robotics research conference in the world, the 2024 edition was held in Yokohama, Japan back in May.

ICRA@40 is not just a second ICRA conference in 2024. Next week’s conference is a single track that promises “a journey through the evolution of robotics and automation,” through four days of short keynotes from prominent roboticists from across the entire field. You can see for yourself, the speaker list is nuts. There are also debates and panels tackling big ideas, like: “What progress has been made in different areas of robotics and automation over the past decades, and what key challenges remain?” Personally, I’d say “lots” and “most of them,” but that’s probably why I’m not going to be up on stage.

There will also be interactive research presentations, live demos, an expo, and more—the conference schedule is online now, and the abstracts are online as well. I’ll be there to cover it all, but if you can make it in person, it’ll be worth it.

Forty years ago is a long time, but it’s not that long, so just for fun, I had a look at the proceedings of ICRA 1984 which are available on IEEE Xplore, if you’re curious. Here’s an excerpt of the forward from the organizers, which included folks from International Business Machines and Bell Labs:

The proceedings of the first IEEE Computer Society International Conference on Robotics contains papers covering practically all aspects of robotics. The response to our call for papers has been overwhelming, and the number of papers submitted by authors outside the United States indicates the strong international interest in robotics.
The Conference program includes papers on: computer vision; touch and other local sensing; manipulator kinematics, dynamics, control and simulation; robot programming languages, operating systems, representation, planning, man-machine interfaces; multiple and mobile robot systems.
The technical level of the Conference is high with papers being presented by leading researchers in robotics. We believe that this conference, the first of a series to be sponsored by the IEEE, will provide a forum for the dissemination of fundamental research results in this fast developing field.

Technically, this was “ICR,” not “ICRA,” and it was put on by the IEEE Computer Society’s Technical Committee on Robotics, since there was no IEEE Robotics and Automation Society at that time; RAS didn’t get off the ground until 1987.

This is a sponsored article brought to you by Khalifa University of Science and Technology.

A total of eight intense competitions to inspire creativity and innovation along with 13 forums dedicated to diverse segments of robotics and artificial intelligence will be part of the 36th edition of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024) in Abu Dhabi.

These competitions at the Middle East and North Africa (MENA) region’s first-ever global conference and exhibition from 14-18 October 2024 at the Abu Dhabi National Exhibition Center (ADNEC) will highlight some of the key aspects of robotics. These include physical or athletic intelligence of robots, remote robot navigation, robot manipulation, underwater robotics, perception and sensing as well as challenges for wildlife preservation.

This edition of IROS is one of the largest of its kind globally in this category because of active participation across all levels, with 5,740 authors, 16 keynote speakers, 46 workshops, 11 tutorials, as well as 28 exhibitors and 12 startups. The forums at IROS will explore the rapidly evolving role of robotics in many industry sectors as well as policy-making and regulatory areas. Several leading corporate majors, and industry professionals from across the globe are gathering for IROS 2024 which is themed “Robotics for Sustainable Development.”

“The intense robotics competitions will inspire creativity, while the products on display as well as keynotes will pave the way for more community-relevant solutions.” —Jorge Dias, IROS 2024 General Chair

Dr. Jorge Dias, IROS 2024 General Chair, said: “Such a large gathering of scientists, researchers, industry leaders and government stakeholders in Abu Dhabi for IROS 2024 also demonstrates the role of UAE in pioneering new technologies and in providing an international platform for knowledge exchange and sharing of expertise. The intense robotics competitions will inspire creativity, while the products on display as well as keynotes will pave the way for more community-relevant solutions.”

The competitions are:

• 2nd AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World? focusing on the physical or athletic intelligence of robots

• The Earth Rovers Challenge on remote robot navigation

• Robotic Construction Challenge and the euROBIN Manipulation Skill Versatility Challenge (MSVC) on robot manipulation

• Underwater Robotics Challenges on innovation and problem-solving in underwater robotics and artificial intelligence

• ROAD-IROS: Automatic Data Annotation Challenge for ROAD Dataset on perception and sensing

• 21st F1Tenth Autonomous Racing Competition

• IEEE RAS Quadruped Robot Challenge (QRC), focusing on navigation

In addition to these competitions, the Falcon Monitoring Challenge (FMC) will focus on advancing the field of wildlife tracking and conservation through the development of sophisticated, noninvasive monitoring systems.

Khalifa University

IROS 2024 will also include three keynote talks on ‘Robotic Competitions’ that will be moderated by Professor Lakmal Seneviratne, Director, Center for Autonomous Robotic Systems (KU-CARS), Khalifa University. The keynotes will be delivered by Professor Pedro Lima, Institute for Systems and Robotics, Instituto Superior Técnico, University of. Lisbon, Portugal; Dr. Timothy Chung, General Manager, Autonomy and Robotics, Microsoft, US; and Dr. Ubbo Visser, President of the RoboCup Federation, Director of Graduate Studies, and Associate Professor of Computer Science, University of Miami, US.

The forums at IROS 2024 will include:

• Robotics in Africa Forum

• Marine Robotics in Ocean Decade Initiative for Sustainable Development

• Robots for Sustainability and Sustainable Robots

• Government Forum: Funding for Robotics Research, Human-Avatars Symbiosis: Can you imagine a future society where you can remotely control multiple avatars?

• Moonshot R&D Program Goal 3 Forum. Envisioning a Future of Human-Robot Co-living: Potential for Robotics to Transform Human Lives

Other forums include:

• Sustainable Medical and Surgical Robotics

• Empowering Diverse Voices in Robotics, Robotics & AI in the UAE: Research Innovation and Entrepreneurship

• Europe Regulates Artificial Intelligence: the Challenge for Robotics, Industrial Opportunities and Socio-Economic Impact of Medical Robotics

• The Future of Work: AI-Enhanced Robotics and Human Interaction Research in M3S

• Robots for a Better Tomorrow: Wellbeing Through Advanced Technology.

One of the largest and most important robotics research conferences in the world, IROS 2024 provides a platform for the international robotics community to exchange knowledge and ideas about the latest advances in intelligent robots and smart machines. A total of 3,344 paper submissions representing 60 countries, have been received from researchers and scientists across the world. China tops the list with more than 1,000 papers, the US with 777, Germany with 302, Japan with 253, and the UK and South Korea with 173 each. The UAE remains top in the Arab region with 68 papers.

One of the largest and most important robotics research conferences in the world, IROS 2024 provides a platform for the international robotics community to exchange knowledge and ideas.

For eight consecutive years since 2017, Abu Dhabi has remained first on the world’s safest cities list, according to online database Numbeo, which assessed 329 global cities for the 2024 listing. This reflects the emirate’s ongoing efforts to ensure a good quality of life for citizens and residents. With a multicultural community, Abu Dhabi is home to people from more than 200 nationalities, and draws a large number of tourists to some of the top art galleries in the city such as Louvre Abu Dhabi and the Guggenheim Abu Dhabi, as well as other destinations such as Ferrari World Abu Dhabi and Warner Bros. World™ Abu Dhabi.

Because of its listing as one of the safest cities, Abu Dhabi continues to host several international conferences and exhibitions. Abu Dhabi is set to host the UNCTAD World Investment Forum, the 13th World Trade Organization (WTO) Ministerial Conference (MC13), the 12th World Environment Education Congress in 2024, and the IUCN World Conservation Congress in 2025.

IROS 2024 is sponsored by IEEE Robotics and Automation Society, Abu Dhabi Convention and Exhibition Bureau, the Robotics Society of Japan (RSJ), the Society of Instrument and Control Engineers (SICE), the New Technology Foundation, and the IEEE Industrial Electronics Society (IES).

More information at https://iros2024-abudhabi.org/

When we think of grasping robots, we think of manipulators of some sort on the ends of arms of some sort. Because of course we do—that’s how (most of us) are built, and that’s the mindset with which we have consequently optimized the world around us. But one of the great things about robots is that they don’t have to be constrained by our constraints, and at ICRA@40 in Rotterdam this week, we saw a novel new Thing: a robotic hand that can detach from its arm and then crawl around to grasp objects that would be otherwise out of reach, designed by roboticists from EPFL in Switzerland.

Fundamentally, robot hands and crawling robots share a lot of similarities, including a body along with some wiggly bits that stick out and do stuff. But most robotic hands are designed to grasp rather than crawl, and as far as I’m aware, no robotic hands have been designed to do both of those things at the same time. Since both capabilities are important, you don’t necessarily want to stick with a traditional grasping-focused hand design. The researchers employed a genetic algorithm and simulation to test a bunch of different configurations in order to optimize for the ability to hold things and to move.

You’ll notice that the fingers bend backwards as well as forwards, which effectively doubles the ways in which the hand (or, “Handcrawler”) can grasp objects. And it’s a little bit hard to tell from the video, but the Handcrawler attaches to the wrist using magnets for alignment along with a screw that extends to lock the hand into place.

“Although you see it in scary movies, I think we’re the first to introduce this idea to robotics.” —Xiao Gao, EPFL

The whole system is controlled manually in the video, but lead author Xiao Gao tells us that they already have an autonomous version (with external localization) working in the lab. In fact, they’ve managed to run an entire grasping sequence autonomously, with the Handcrawler detaching from the arm, crawling to a location the arm can’t reach, picking up an object, and then returning and reattaching itself to the arm again.

Today, Boston Dynamics and the Toyota Research Institute (TRI) announced a new partnership “to accelerate the development of general-purpose humanoid robots utilizing TRI’s Large Behavior Models and Boston Dynamics’ Atlas robot.” Committing to working towards a general purpose robot may make this partnership sound like a every other commercial humanoid company right now, but that’s not at all that’s going on here: BD and TRI are talking about fundamental robotics research, focusing on hard problems, and (most importantly) sharing the results.

The broader context here is that Boston Dynamics has an exceptionally capable humanoid platform capable of advanced and occasionally painful-looking whole-body motion behaviors along with some relatively basic and brute force-y manipulation. Meanwhile, TRI has been working for quite a while on developing AI-based learning techniques to tackle a variety of complicated manipulation challenges. TRI is working toward what they’re calling large behavior models (LBMs), which you can think of as analogous to large language models (LLMs), except for robots doing useful stuff in the physical world. The appeal of this partnership is pretty clear: Boston Dynamics gets new useful capabilities for Atlas, while TRI gets Atlas to explore new useful capabilities on.

Here’s a bit more from the press release:

The project is designed to leverage the strengths and expertise of each partner equally. The physical capabilities of the new electric Atlas robot, coupled with the ability to programmatically command and teleoperate a broad range of whole-body bimanual manipulation behaviors, will allow research teams to deploy the robot across a range of tasks and collect data on its performance. This data will, in turn, be used to support the training of advanced LBMs, utilizing rigorous hardware and simulation evaluation to demonstrate that large, pre-trained models can enable the rapid acquisition of new robust, dexterous, whole-body skills.

The joint team will also conduct research to answer fundamental training questions for humanoid robots, the ability of research models to leverage whole-body sensing, and understanding human-robot interaction and safety/assurance cases to support these new capabilities.

For more details, we spoke with Scott Kuindersma (Senior Director of Robotics Research at Boston Dynamics) and Russ Tedrake (VP of Robotics Research at TRI).

How did this partnership happen?

Russ Tedrake: We have a ton of respect for the Boston Dynamics team and what they’ve done, not only in terms of the hardware, but also the controller on Atlas. They’ve been growing their machine learning effort as we’ve been working more and more on the machine learning side. On TRI’s side, we’re seeing the limits of what you can do in tabletop manipulation, and we want to explore beyond that.

Scott Kuindersma: The combination skills and tools that TRI brings the table with the existing platform capabilities we have at Boston Dynamics, in addition to the machine learning teams we’ve been building up for the last couple years, put us in a really great position to hit the ground running together and do some pretty amazing stuff with Atlas.

What will your approach be to communicating your work, especially in the context of all the craziness around humanoids right now?

Tedrake: There’s a ton of pressure right now to do something new and incredible every six months or so. In some ways, it’s healthy for the field to have that much energy and enthusiasm and ambition. But I also think that there are people in the field that are coming around to appreciate the slightly longer and deeper view of understanding what works and what doesn’t, so we do have to balance that.

The other thing that I’d say is that there’s so much hype out there. I am incredibly excited about the promise of all this new capability; I just want to make sure that as we’re pushing the science forward, we’re being also honest and transparent about how well it’s working.

Kuindersma: It’s not lost on either of our organizations that this is maybe one of the most exciting points in the history of robotics, but there’s still a tremendous amount of work to do.

What are some of the challenges that your partnership will be uniquely capable of solving?

Kuindersma: One of the things that we’re both really excited about is the scope of behaviors that are possible with humanoids—a humanoid robot is much more than a pair of grippers on a mobile base. I think the opportunity to explore the full behavioral capability space of humanoids is probably something that we’re uniquely positioned to do right now because of the historical work that we’ve done at Boston Dynamics. Atlas is a very physically capable robot—the most capable humanoid we’ve ever built. And the platform software that we have allows for things like data collection for whole body manipulation to be about as easy as it is anywhere in the world.

Tedrake: In my mind, we really have opened up a brand new science—there’s a new set of basic questions that need answering. Robotics has come into this era of big science where it takes a big team and a big budget and strong collaborators to basically build the massive data sets and train the models to be in a position to ask these fundamental questions.

Fundamental questions like what?

Tedrake: Nobody has the beginnings of an idea of what the right training mixture is for humanoids. Like, we want to do pre-training with language, that’s way better, but how early do we introduce vision? How early do we introduce actions? Nobody knows. What’s the right curriculum of tasks? Do we want some easy tasks where we get greater than zero performance right out of the box? Probably. Do we also want some really complicated tasks? Probably. We want to be just in the home? Just in the factory? What’s the right mixture? Do we want backflips? I don’t know. We have to figure it out.

There are more questions too, like whether we have enough data on the Internet to train robots, and how we could mix and transfer capabilities from Internet data sets into robotics. Is robot data fundamentally different than other data? Should we expect the same scaling laws? Should we expect the same long-term capabilities?

The other big one that you’ll hear the experts talk about is evaluation, which is a major bottleneck. If you look at some of these papers that show incredible results, the statistical strength of their results section is very weak and consequently we’re making a lot of claims about things that we don’t really have a lot of basis for. It will take a lot of engineering work to carefully build up empirical strength in our results. I think evaluation doesn’t get enough attention.

What has changed in robotics research in the last year or so that you think has enabled the kind of progress that you’re hoping to achieve?

Kuindersma: From my perspective, there are two high-level things that have changed how I’ve thought about work in this space. One is the convergence of the field around repeatable processes for training manipulation skills through demonstrations. The pioneering work of diffusion policy (which TRI was a big part of) is a really powerful thing—it takes the process of generating manipulation skills that previously were basically unfathomable, and turned it into something where you just collect a bunch of data, you train it on an architecture that’s more or less stable at this point, and you get a result.

The second thing is everything that’s happened in robotics-adjacent areas of AI showing that data scale and diversity are really the keys to generalizable behavior. We expect that to also be true for robotics. And so taking these two things together, it makes the path really clear, but I still think there are a ton of open research challenges and questions that we need to answer.

Do you think that simulation is an effective way of scaling data for robotics?

Tedrake: I think generally people underestimate simulation. The work we’ve been doing has made me very optimistic about the capabilities of simulation as long as you use it wisely. Focusing on a specific robot doing a specific task is asking the wrong question; you need to get the distribution of tasks and performance in simulation to be predictive of the distribution of tasks and performance in the real world. There are some things that are still hard to simulate well, but even when it comes to frictional contact and stuff like that, I think we’re getting pretty good at this point.

Is there a commercial future for this partnership that you’re able to talk about?

Kuindersma: For Boston Dynamics, clearly we think there’s long-term commercial value in this work, and that’s one of the main reasons why we want to invest in it. But the purpose of this collaboration is really about fundamental research—making sure that we do the work, advance the science, and do it in a rigorous enough way so that we actually understand and trust the results and we can communicate that out to the world. So yes, we see tremendous value in this commercially. Yes, we are commercializing Atlas, but this project is really about fundamental research.

What happens next?

Tedrake: There are questions at the intersection of things that BD has done and things that TRI has done that we need to do together to start, and that’ll get things going. And then we have big ambitions—getting a generalist capability that we’re calling LBM (large behavior models) running on Atlas is the goal. In the first year we’re trying to focus on these fundamental questions, push boundaries, and write and publish papers.

I want people to be excited about watching for our results, and I want people to trust our results when they see them. For me, that’s the most important message for the robotics community: Through this partnership we’re trying to take a longer view that balances our extreme optimism with being critical in our approach.

Simone Giertz came to fame in the 2010s by becoming the self-proclaimed “queen of shitty robots.” On YouTube she demonstrated a hilarious series of self-built mechanized devices that worked perfectly for ridiculous applications, such as a headboard-mounted alarm clock with a rubber hand to slap the user awake.

But Giertz has parlayed her Internet renown into Yetch, a design company that makes commercial consumer products. (The company name comes from how Giertz’s Swedish name is properly pronounced.) Her first release, a daily habit-tracking calendar, was picked up by prestigious outlets such as the Museum of Modern Art design store in New York City. She has continued to make commercial products since, as well as one-off strange inventions for her online audience.

Where did the motivation for your useless robots come from?

Simone Giertz: I just thought that robots that failed were really funny. It was also a way for me to get out of creating from a place of performance anxiety and perfection. Because if you set out to do something that fails, that gives you a lot of creative freedom.

You built up a big online following. A lot of people would be happy with that level of success. But you moved into inventing commercial products. Why?

Giertz: I like torturing myself, I guess! I’d been creating things for YouTube and for social media for a long time. I wanted to try something new and also find longevity in my career. I’m not super motivated to constantly try to get people to give me attention. That doesn’t feel like a very good value to strive for. So I was like, “Okay, what do I want to do for the rest of my career?” And developing products is something that I’ve always been really, really interested in. And yeah, it is tough, but I’m so happy to be doing it. I’m enjoying it thoroughly, as much as there’s a lot of face-palm moments.

Giertz’s every day goal calendar was picked up by the Museum of Modern Art’s design store. Yetch

What role does failure play in your invention process?

Giertz: I think it’s inevitable. Before, obviously, I wanted something that failed in the most unexpected or fun way possible. And now when I’m developing products, it’s still a part of it. You make so many different versions of something and each one fails because of something. But then, hopefully, what happens is that you get smaller and smaller failures. Product development feels like you’re going in circles, but you’re actually going in a spiral because the circles are taking you somewhere.

What advice do you have for aspiring inventors?

Giertz: Make things that you want. A lot of people make things that they think that other people want, but the main target audience, at least for myself, is me. I trust that if I find something interesting, there are probably other people who do too. And then just find good people to work with and collaborate with. There is no such thing as the lonely genius, I think. I’ve worked with a lot of different people and some people made me really nervous and anxious. And some people, it just went easy and we had a great time. You’re just like, “Oh, what if we do this? What if we do this?” Find those people.

This article appears in the November 2024 print issue as “The Queen of Useless Robots.”

Donald Trump will embrace the truth Joe Biden has refused to countenance: Israel's enemies are America's enemies. And when Israel defeats its enemies, America wins.

As we share our truths and witness each other's, we build unity and community.

Eliminating this pernicious policy should be on the Trump administration's first week to-do list.

Friday on the RealClearPolitics radio show -- weeknights at 6:00 p.m. on SiriusXM's POTUS Channel 124 and then on Apple, Spotify, and here on our website -- Tom Bevan, Andrew Walworth, and Carl Cannon look at the latest count to determine the final popular vote and House majority.

It's hard to imagine opposing Trump's proposal. Who would want to help murderers and drug dealers who entered the country illegally remain in the United States?

The most impressive aspect of Donald Trump's victory over Vice President Kamala Harris last week was the uniformity of his gains across the electoral landscape.

Other Democrats lambasted him. The Tufts political science department spurned him. But Moulton is raising concerns the left needs to take seriously.

November and December will nominally be about confirming judges and kicking the can on must-pass bills. More ambitious efforts probably aren't happening.

Upon further review, analyst Sean Trend may be right: There may be an emerging GOP majority nationwide.

Senate Republicans cannot be led by someone who is openly hostile to the agenda of their party's president and the base who elected him.

The election is over and the economy had a huge impact. An AP analysis said 96% of those surveyed admitted that prices of gas and groceries had an influence on their vote.

Meta had huge success last year with its flagship Quest 3 VR headset and it is back with a significantly cheaper 3S device that compromises on visuals but still delivers a great experience

From Goats to airplanes, Xbox Game Pass has another bumper month in store for subscribers. Here's everything you need to know about what is heading to PC and console this November 2024

Aside from offering a more powerful console, the PS5 Pro also packs a sneaky theme for PlayStation fans to uncover - something Sony hadn't previously discussed.

Death Stranding, originally launched in 2019, has finally come to Xbox consoles five years after its initial debut as a Director's Cut version from Hideo Kojima.

PlayStation 5 Pro is out, and with plenty of availability, scalpers are shifting the £700 console at a loss and turning their attention to a key accessory instead

Destiny 2's Exotic vendor, Xur, is back again this week. Here's how to find him, what he's selling, and whether we recommend his offerings before he vanishes again.

EA FC 25 players can snag some big freebies this week, with EA Sports celebrating the Ballon d'Or in style for all Ultimate Team players with some of the best players around.

Black Ops 6 is here, and fans have been itching to know what's included in Season 1 - and now we have an answer. Here's what's included, and when you can play

PS5 owners can play some great games this month, including a Bethesda hit and more. Here's what's free for PS Plus subscribers for November, with more to be announced.

Call of Duty fans are impressed by Black Ops 6 Season 1 already, calling it the 'best Season 1 roadmap' since 2019's Modern Warfare reboot and you can find out below why they're saying it

The Pokemon Company is teaming up with content creators to stage a 24-hour unboxing live stream for its new Scarlet & Violet - Surging Sparks card set as you Gotta Catch 'Em All

One fan has been rebuilding Final Fantasy 8 but it's not going quickly enough for some people who suggested they cut out things like relationships to focus on the game.

Just got a PS5 or want to know what to prioritise in your backlog? We've got you covered with the best PlayStation 5 games for your console that you can play right now.

Pikachu ex, found in the new Pokemon TCG set Surging Sparks, has seen its value soar with players spending over $500 for the latest card. Here's why.

Fortnite fans are going wild after Epic Games announced it's bringing back the original version of its battle royale as Fortnite OG as a permanent addition to the game.

Scientists say geoengineering, or doing things like intentionally increasing Earth’s reflectivity or blocking the sun, is a “really big deal” in slowing down climate change. Here are the ideas they are proposing.

More than 80 per cent of B.C.'s water basins are experiencing level 4 or 5 drought conditions, with salmon in many parts of the province struggling to make it to their spawning grounds.

Fisheries and Oceans Canada whale researchers recently spotted one of the largest pods of orca whales ever reported off the coast of Newfoundland and Labrador.

Certain strains of E. coli are capable of causing severe disease, by rapidly spreading through the human digestive system, wreaking havoc throughout the bloodstream, and eventually damaging the delicate kidneys. That's the situation right now during a large outbreak in Alberta, with hundreds of children now affected.

NASA said Thursday that the study of UFOs will require new scientific techniques, including advanced satellites as well as a shift in how unexplained sightings are perceived.

Two large grizzly bears followed 13 hikers down a trail in Banff National Park for 20 minutes — with one even making a few quick runs at the group.

In this week's issue of our environment newsletter, we look at the carbon capture potential of the world's oceans and what effect beavers are having in the Arctic (spoiler: it's not good).

Researchers calculated that if 30 per cent of vehicles in Chicago currently running on combustion engines were converted to electric, the reduction in pollution would save billions in health care costs every year. 

Online group Girls Who Chase has created a global community of women who head into severe weather to record images, report damage and help scientists understand the impact of storms to be better prepared