👽 FiToSci October 2021: Meta Headsets, Robot Dog Sniper, Ultra Cruise, Xenotransplantation, Orbital Reef, and More
Hey there, I'm Emil Protalinski. This is FiToSci, a monthly newsletter that tracks how humanity is taking the fiction out of science fiction. Make sure to star this email to easily reference October's highlights:
🕶️ Augmented/virtual reality: Meta teased new AR/VR headsets.
🤖 AI/robots: Ghost Robotics gave its robot dog a Sword sniper rifle.
🚗 Transportation/logistics: GM extended its self-driving system 10x.
🧬 Biotech/bioscience: Scientists attached a pig kidney to a human.
🚀 Space: Blue Origin and Sierra Space announced a space station.
I recommend skipping to the sections you like, opening what you find interesting in a separate tab, and bookmarking links for later reading. Let's get started.
🕶️ Facebook rebranded as Meta, affirming its all-in bet on building the metaverse that originated from Snow Crash and Ready Player One, and teased two upcoming AR/VR devices. Project Cambria, a high-end mixed reality wireless headset previously leaked as Quest Pro, is coming next year with cameras that support high-resolution full-color video passthrough. Project Nazare, meanwhile, is the codename for Meta's first consumer AR glasses, which are still "a few years out." To keep developers engaged in the interim, the company debuted Presence Platform that builds on the Passthrough API released in July with three SDKs to create mixed reality experiences for the Quest VR headset.
🕶️ Meta also announced a slew of software products, including a social VR space called Horizon Home, Messenger VR calls, Quest for Business, and progressive web apps in the Oculus Store that bring 2D internet services into the metaverse. The company wants people to work in VR so badly that Quest for Business, scheduled for early testing later this year, beta in 2022, and full availability in 2023, will let users log into a Quest 2 with a work account instead of a personal Facebook account.
🕶️ Light Field Lab debuted SolidLight, its new high-resolution holographic display shipping as early as next year. In a 28-inch display space, SolidLight can generate holograms of 2.5 billion pixels with a density of 10 billion pixels per meter, but the company is envisioning holographic video walls with upward of 245 billion pixels, reminiscent of Star Trek's holodeck.
🕶️ Pimax unveiled Pimax Reality 12K QLED, a $2,400 VR headset shipping in Q4 2022 that can be used as a standalone headset or PC VR device. The headset supports nearly 6K resolution per eye at 200Hz, a 200-degree field of view, body tracking, eye tracking, dynamic foveated rendering, and split rendering, which lets the headset leverage the cloud via 5G for improved VR rendering and processing power.
🕶️ Cisco unveiled Webex Hologram, an AR meeting collaboration product that supports Magic Leap and Microsoft's HoloLens headsets. Available to a limited number of customers in a pilot program, Webex Hologram promises real-time holograms of participants, enabling them to share physical and digital content.
🕶️🤖 Google's Jigsaw announced Trainer, a VR police training simulator built on a conversational AI tool. Each AI character in a Trainer scenario responds in real-time to the user's language and behavior to teach de-escalation techniques based on input from civil society organizations, academics, researchers, civil rights activists, and law enforcement.
🕶️🤖 Meta (formerly Facebook) detailed Ego4D, an AI/AR research project in partnership with 13 universities that uses first-person video footage to help AI assistants and AR tools understand the world as humans do. The researchers listed five skills it wants these systems to develop: episodic memory ("Where did I leave my keys?"), forecasting ("Wait, you’ve already added salt to this recipe?"), hand and object manipulation ("Teach me how to play the drums"), audio-visual diarization ("What was the main topic during class?"), and social interaction ("Help me better hear the person talking to me at this noisy restaurant").
🤖 Ghost Robotics outfitted its robot dog with a sniper rifle, calling it "the latest lethality innovation." Sword's Special Purpose Unmanned Rifle is designed for precision fire from unmanned platforms and features day and night sensors with a range of 3,900 feet. While the robodog requires a human to control the weapon, and we already have remotely piloted robotic ground weapons (not to mention killer drones), this seems like an inflection point in developing killer robots. International laws do not currently prohibit autonomous weapons systems, but we may want to accelerate straight towards Isaac Asimov's three laws of robotics.
🤖 Researchers developed Leonardo, a two-legged flying robot inspired by Astro Boy that can leap over obstacles, skateboard, and balance on a slackline. While Leonardo is just 75cm tall and weighs only 2.58kg, a full-fledged version could be tasked with inspecting and repairing damaged infrastructure, installing new equipment in hard-to-reach places, aiding in natural disasters and industrial accidents, or even exploring celestial bodies.
🤖 Researchers taught a robotic mini cheetah to hop over uneven terrain in real-time by processing input from a front-mounted camera and translating information from another camera into instructions for moving its body. By relying on a neural network rather than pre-mapped terrain, robot quadrupeds could one day sprint into the woods on emergency response missions or climb flights of stairs to deliver medication to housebound individuals.
🤖 Researchers developed a neural network that can detect shadows and reflections invisible to the human eye from a video of a surface to determine the number of people in a room with 94.4% accuracy and what those people are up to with 97.3% accuracy. Even James Bond would be jealous of this spy tech that figures out what someone avoiding a camera's line of sight is doing.
🤖 Researchers trained a walking algorithm for the doglike robot ANYmal in less than one-hundredth of the time it normally takes using reinforcement learning on 4,000 virtual versions of the robot. The simulation helped ANYmals overcome obstacles like steps, slopes, and sharp drops carved into a virtual landscape, although the robots still had issues at higher speeds in the real world.
🤖🚗 Moscow announced Face Pay, the world's first facial recognition fare payment system, rolling out across more than 240 metro stations. The cashless, cardless, and phone-less system asks passengers to connect their photo, bank card, and transit card through the Moscow Metro's mobile app, raising privacy and surveillance concerns.
🤖🚗 Researchers developed a neural network that lets drones fly autonomously up to 40 km/h, without a map, through complicated environments like forests, buildings, caves, ruins, and trains, all without crashing into obstacles. The high-speed drone flights required only onboard cameras and computation, which could be useful in emergencies, construction sites, security applications, or even the exploration of other planets.
🤖🚗 Researchers trained a deep learning traffic model on historical crash data, road maps, satellite imagery, and GPS traces to identify high-risk areas and predict future car accidents. The model offers high-resolution crash risk maps describing the expected number of crashes over time, which could help city planners design safer roads and insurance companies provide custom plans based on driving trajectories.
🚗 General Motors unveiled Ultra Cruise, a driver-assist system meant to cover over 2 million miles of U.S. and Canadian roads at launch in 2023 and ultimately provide a full hands-free driving experience in 95% of driving scenarios. Ultra Cruise is supposed to offer 10-times the coverage of Super Cruise, which is limited to 200,000 miles of U.S. and Canadian highway, by spanning major highways, paved rural roads, city streets, and subdivisions. The Level 2 self-driving system would change lanes on demand and automatically; it would also make left- and right-hand turns, avoid close objects, navigate stop signs and traffic signals, and park in residential driveways.
🚗 Tesla expanded its Full Self Driving (FSD) beta program, which offers Level 2 autonomous features, to owners who request access and have the best safety score, per Tesla's analysis. For the past 12 months, Tesla has been testing the FSD beta program with about 2,000 people while the "wider release" was delayed multiple times.
🚗 Researchers deployed a full-scale version of the fully autonomous boat Roboat on the canals of Amsterdam. After multiple redesigns since the research project began in late 2015, the battery-powered Roboat is now ready to start transporting goods, people, and waste, and extend shorelines and form floating bridges.
🚗 XPeng unveiled a flying car coming in 2024 that can also drive on roads thanks to its foldable rotors. While the company noted that the design of the electric take-off and landing (eVTOL) vehicle might change, it certainly looks like the AirCar that completed its inaugural inter-city flight in June.
🚗 Air unveiled Air One, a two-seater eVTOL vehicle that it aims to sell directly to consumers in 2024. Air expects to obtain certification by the end of 2023 for its flying car with a 110-mile flight range, 155-mph cruising speed, collapsible wings, a glass bottom, and an AI monitoring system that performs safety inspections.
🚗 Jetson Aero unveiled Jetson One, a $92,000 single-seat eVTOL slated to arrive next fall with a maximum speed of 102 km/h and hover time of 20 minutes on a single charge. Jetson One weighs 86kg and can supposedly carry a passenger weighing up to 96kg, a claim that didn't stop the limited 2022 production of 12 units from selling out.
🚗 ALI Technologies launched the XTurismo Limited Edition, a $680,000 motorcycle-like hoverbike powered by an internal combustion engine and electric motors that let it fly up to 40 minutes at a maximum speed of 100 km/h. Slated to ship in the first half of next year, the company expects consumers to use XTurismo at race tracks and rescue teams to use the hoverbike to access difficult-to-reach areas.
🚗 Wing announced that Walgreens employees would use delivery drones to fulfill orders in the Dallas-Fort Worth, Texas area. This is the first time the Alphabet company, which in August passed 100,000 drone deliveries, will use a third party to load up the drones with packages, and the first time its vehicles will be used in a densely populated market.
🚗🧬 University Health Network and Unither Bioelectronics announced the world's first successful lung delivery by drone, which was transported in a carbon fiber container with an emergency parachute and GPS. After 18 months of practice flights, dummy packages, and drop tests, the pair of human lungs flew for six minutes between hospitals before being transplanted into a 63-year-old engineer and drone enthusiast with pulmonary fibrosis.
🧬 Scientists temporarily attached a pig kidney to a human, and it worked. The pig in question had been genetically engineered to remove a sugar molecule that causes humans to immediately reject pig organs, while the human patient was brain dead. The organ was connected to the patient's blood vessels in the upper leg, outside the abdomen, where it functioned normally for two days. If scientists can verify the longevity of such organs and determine the risk of long-term rejection is unlikely, gene editing and xenotransplantation (the process of grafting or transplanting body parts between species) could save the lives of those on backlogged organ waitlists.
🧬 Scientists depleted oxygen in water housing tadpoles, injected the tadpoles with photosynthetic algae that produce oxygen, and then shined a light on the new hybrids to confirm they could survive without any environmental oxygen. The biological experimentation harnessing photosynthesis in animals could have important medical applications for oxygen deprivation in humans.
🧬 Researchers implanted ultra-thin, tissue-friendly microelectrodes into rat brains to block pain signals from reaching the cerebral cortex without affecting any other sensory system or motor skill. Side-effect-free pain relief has been a major challenge until now, and the team hopes that within five to eight years the technology could scale to treat human pain and maybe even degenerative brain diseases.
🧬 Researchers genetically modified barley plants to grow meat. The altered barley is harvested and purified to extract growth factor proteins, which stimulate the growth of tissue, muscle, and fat cells for lab-grown meat without relying on animals.
🧬 Scientists implanted a visual neuroprosthesis in the brain of a 57-year-old woman, who had been completely blind for more than 16 years to directly stimulate her visual cortex to see shapes and letters. The Star Trek-like artificial vision system turns light detected using a pair of glasses into electrical signals sent to the prosthesis' 3D matrix of 96 microelectrodes, completely bypassing the eyes.
🧬🚀 Researchers proposed producing rocket fuel on Mars using microbes from Earth, potentially avoiding the need to bring enough gas for the return trip. The bioproduction concept would leverage carbon dioxide, frozen water, and sunlight from Mars, along with cyanobacteria (blue-green algae) to gather CO2 from the atmosphere and a bioengineered strain of E. coli bacteria to convert the sugar created by the algae into a viable propellant.
🚀 Blue Origin and Sierra Space announced plans for Orbital Reef, a commercial space station to be built in low Earth orbit with help from Boeing, Redwire Space, Genesis Engineering Solutions, and Arizona State University. Blue Origin describes Orbital Reef as a "mixed-use business park" that would be habitable for up to 10 people as well as support spacecraft and modules docking at multiple ports. Potential customers for the station include space agencies, countries without space programs, and travel companies. Orbital Reef is expected to deploy between 2025 and 2030 with the goal of picking up some of the International Space Station's business following its retirement.
🚀 Nanoracks, Lockheed Martin, and Voyager Space announced Starlab, the "first-ever" commercial space station slated to be operational by 2027. Designed to host four astronauts, Starlab would feature an expansive inflatable habitat, a metallic docking node, a power and propulsion element, a large robotic arm for servicing cargo and payloads, and a laboratory.
🚀 Blue Origin successfully sent four passengers to space, including William Shatner, best known for playing Captain Kirk in Star Trek. At 90 years old, Shatner became the oldest human in space, taking the record from 82-year-old Wally Funk, who Blue Origin sent up in July.
🚀 Physicists proposed Pulverize It, a planetary defense system that would use rocket-launched interceptors deployed ahead of an incoming asteroid or comet to break it into fragments that would then disintegrate and burn in Earth's upper atmosphere. Asteroids greater than a kilometer wide could be fragmented using barrages of nonnuclear interceptors launched from Earth or in its vicinity, while larger threats would require small nuclear penetrators.
🚀 Scientists found, according to a computer simulation, that nuking an incoming asteroid months before impact could work depending on the size. A deflection strategy that completely avoids impact is preferable to the disruption approach, but maybe it's reassuring to know Armageddon was onto something.
🚀 NASA shared the complex series of steps required to bring the first samples of Mars rock to Earth, which the Perseverance rover is currently drilling and setting aside for another mission later this decade. That mission would entail a rocket launching a spacecraft from Earth to Mars, a lander setting down onto the surface, a fetch rover collecting the sealed samples, an ascent vehicle launching the sample container into Mars orbit, and a retrieval spacecraft with a payload for transferring the Earthbound sample capsule.
And that's October. As a treat for reading right to the very end, check out NASA's Visions of the Future video.
Feel free to reply to this email with comments or questions; I respond to every reply. Follow FiToSci on Twitter and forward this issue to a friend who would enjoy it. See you next month!