Drawing of a Magic Lantern / The first film projectors were called Magic Lanterns
“We’re storytellers, rocket scientists, wizards, gurus and more. And we’re here to make magic real for you.” Magic Leap Website
“Maybe its magic is raising over a billion dollars from investors?” One skeptical observer
The world of technology has created many magic devices but the future of digital technology might be in creating magic environments rather than magic devices. Many leading tech companies are betting their futures on creating these magic environments. None more than the world’s greatest tech company Google.
In the annual founder’s letter to shareholders, Google CEO Sundar Pichai writes, “Looking to the future, the next big step will be for the very concept of the ‘device’ to fade away.” The fate of the computer will move from being device to service assistant. As Pichai notes, “Over time, the computer itself – whatever its form factor – will be an intelligent assistant helping you through your day.” Instead of online information and activity happening mostly on the rectangular touch screens of smartphones, Pichai sees artificial intelligence powering increasingly formless computers. “We will move from mobile first to an AI first world,” he said.
Google’s most noticeable venture into AI was with Google Glass in 2013 that attempted to place this “intelligent assistant” in eyeglasses. As Google co-founder Sergey Brin observed at the time, summoning information to one’s eyes is a better use of the body than “rubbing a featureless piece of glass.”
But the world was not yet ready to stop rubbing that “featureless piece of glass” and Google Glass was a failure for a number of reasons such as privacy concerns and technological problems. Ironically, though, perhaps the biggest reason for the failure of Google Glass was that it was too much of that “device” Sunda Pichai sees fading away.
In the three years since the failure of Google Glass, Google’s pursuit of artificial intelligence has continued in other areas. Today, the flagship for Google’s pursuit of AI is a secretive Florida company called Magic Leap. Google is a major investor in Magic Leap and Pichai sits on the Magic Leap board. The company has raised more than $1 billion to build a system that inserts 3-D moving images and other information into the surroundings that people already see.
Magic Leap calls their new technology mixed reality to distinguish it from virtual reality and augmented reality. To see some stunning images of this new technology, one doesn’t have to go any further than the home page for Magic Leap. Here, we’re presented with a number of short clips demonstrating the technology of Magic Leap in action. The first scene that everyone who visits the website now sees is (I think important symbolically and intentionally) set within an elementary school setting. There is important subtext already at work on the viewer, the image provided by the company, suggesting what few have ever suggested before: that the company’s technology might have a large – grand – use in not just entertainment but also in education. The short video proves their point in a powerful manner. Within seconds, the usual school day of the learned, dull, drudgery and boredom, is suddenly crashed through like the image through the floor of the basketball court. The old methods of education and teaching in America, like the wood on the basketball court, smashed to pieces within the first few seconds of the video.
• Virtual Reality (VR) – Digital environments that shut out the real world. VR places the user in another location entirely. Whether that location is computer-generated or captured by video, it blocks off the user’s natural surroundings.
• Augmented Reality (AR) – Digital content on top of your real world. In augmented reality – like Google Glass or the Yelp app’s Monocle feature on mobile devices – the visible natural world is overlaid with a layer of digital content.
• Mixed Reality (VR) – Digital content that reacts to your real world. In technologies like Magic Leap’s, virtual objects are integrated into – and responsive to – the natural world. A virtual ball under your desk, for example, would be blocked from view unless you bent down to look at it. In theory, MR could become VR in a dark room.
Three Types of Artificial Realities
Interestingly, when most attention has been focused on virtual reality, the true revolutionary technology of artificial intelligence might really be mixed reality.
Rony Abovitz and his magic chip
Rony Abovitz founded the secretive company in 2010. It is not located where one might suspect in the heart of Silicon Valley but rather in the beach town of Dania, Florida just south of Ft. Lauderdale and a few miles west of the Atlantic Ocean.
He’s not what one would expect of a typical high tech entrepreneur. As a kid growing up in South Florida, Abovitz was enthralled by science fiction, music and robots and gravitated toward robots as a career. From 1988 – 1996 he attended the University of Miami and in 1996 received a Masters in Biomedical Engineering. He was the weekly cartoonist for the school paper (The Miami Hurricane), DJ for WVUM 90.5, member of UM’s varsity track and field team where he threw the javelin and a member of Tau Beta Pi fraternity. Most people found Abovitz’s cartoons more weird than funny. They were stream-of-consciousness doodles featuring alien creatures, annotated by tiny inscriptions that include secret messages to girlfriends. They did not appear to come from the mind of an engineer.
One of Abovitz’s Cartoons at UM
In 1997, a year after graduating from UM, he founded a company called Z-Cat and spun out the robotics group of the company to create Mako Surgical Corporation that built robots for surgery. It became a publicly traded medical device company that manufactures and markets surgical robotic arm assistance platforms. Their most notable product was the RIO (Robotic Arm Interactive Orthopedic System) as well as orthopedic implants used by orthopedic surgeons for use in partial knee and total hip arthroplasty. They became known for their intellectual property of devices and have over 300 U.S. and foreign patents and patent applications. The company won numerous awards, including being named the fastest growing technology company in 2011 on Deloitte’s Technology Fast 500. On September 25, 2013, the Board of Directors of Mako Surgical accepted a deal to merge with Stryker Medical for $1.65B. The deal closed in December 2013.
* * *
But Abovitz had become active in other endeavors by the time Mako was sold. In 2010, he founded Magic Leap Studios, the precursor to Magic Leap. Magic Leap Studios was more of a special effects studio than a tech firm. It was working on a graphic novel and a feature film series and had a relationship with Weta Workshop, the company behind the amazing props and creatures in Lord of the Rings, The Hobbit and District 9. Augmented reality didn’t seem to be the goal at the time. The prominent comic book duo Dan Abnett and Andy Lanning had signed on to help flesh out the idea.
By 2011, Magic Leap was morphing more into a tech company rather than a film studio. At Comic-Con that July, it released an augmented reality app called Hour Blue, a first step towards a new goal to “bring cinema into the physical world.” The prominent comic book duo Dan Abnett and Andy Lanning had signed on to help flesh out the idea.At the 2011 Comic-Con, Abovitz is captured on video explaining his concept for those who might be able to decipher it.
In December of 2012, Abovitz gave one of the strangest TED Talks in the history of the technology talks. Delivering something that was more performance art than conference speech, it began with two furry green and pink monsters called Shaggles, who started faux-fighting to the “Space Odyssey” theme song. Eventually, Abovitz joined them onstage in a spacesuit. “A few awkward steps for me,” he said flatly, “a magic leap for mankind.” Behind him, a live band erupted into a clash of sounds and screams, singing, and the Shaggles started throwing around cards bearing the word “fudge” and dancing wildly. The Abovitz TED Talk can be viewed on YouTube.
My two cents worth: it did not receive the greatest reviews that day at the TED conference judging from the silence in the TED audience after the performance. Yet, again, I think that one can make something symbolically around this short little weird video. First of all, it begins with the broadest topics and realities possible. Outer space of a galaxy and then the planets of a particular solar system and then, a particular planet in this solar system and raging clouds showing a certain power and majesty of nature. Then, the apes in their costumes fighting over a piece of fudge on stage. A person staring through a hole at the top right of the stage kike she is staring down at all of us in some type of box. Certainly, a strange, unique little episode in the history of TED talks. But again, the symbolism of the entire film: the two images clashing together, the majesty of dreams and far away, visionary thoughts, mixed with all the stupidity and playfulness in the world. A vision that seemed to sum up the life of Rony Abovitz pretty well so far. The majesty of space and distant dreams mixed with the always humorous reality of the world. It was a message not hard to take away from the short little, enigmatic performance at the TED Talk.
Sparky Dog & Friends
The weird theatrics of Abovitz fit in pretty well with a rock band Abovitz was in/is in called Sparky Dog & Friends. Abovitz had never abandoned his childhood love of music has never abandoned his love of music. Sparky Dog & Friends is comprised of Abovitz, his wife Debb, Arthur Quaid, the former MAKO Surgical director of robotics and Graham MacNamara, Magic Leap patent holder. Their music is not bad either. (Hear some of Sparky Dog & Friends music on Soundcloud. Pretty damn good music I think being a musician myself. Something close to, yes I’ll say it, close to the sound of that great band Stereo Lab. It is the kind of music that could serve as a powerful music for the a story that I think would make an amazing screenplay, film or television series. Based around the life of a Rony Abovitz character. The background music to the story of course would be from Sparky Dog & Friends. Anyway, you be the judge after completing the rest of this of this article.)
He seemed to possess an unusual mixture of interests and experiences somehow all contained within one person, all held together somehow. A wealthy inventor of a leading robotic surgical company. A rock musician. A performance artist. A new type of filmmaker. A former college cartoonist. An innovator in one of the greatest new medical technologies of the past fifty years. Medical robotics in surgery. What new direction would Abovitz chart out next? Listening to Sparky Dog & Friends and watching Abovitz’s performance at the TED Talk are not confidence inspiring things. But Abovitz had never changed much from that original way he was.
But he was such a young man split so much more than most young men. The ideas of the small, non-important things of culture, the toys of culture, the not-so-serious-things of culture. Yet against these small, everyday things, rubbed the concerns for the major things of life. His interest in artificial intelligence. His almost worship of it. No matter how one ultimately defines Abovitz, the words “artificial intelligence” have to be attached somehow.
* * *
Yet a only a year after the TED Talk, the company had gone from Magic Leap Studios to just Magic Leap and had become the darling of Google and a host of other major tech players. And, unlike Mako and its robotic surgical arm, Magic Leap had yet to create one product.
The year 2014 was a big year for Magic Leap. In February of 2014, VentureBeat reported the mysterious firm had secured $50 million to build a “game changing” technology that would “forever change the way we interact with images and information,” in the company’s words. In March of 2014, Magic Leap got another huge shot of credibility when veteran game designer Graeme Devine signed on to build an entire game studio around the mystery tech. In April of 2014, veteran tech marketing guru Brian Wallace joined. Wallace is one of the men responsible for Samsung’s wildly successful “Next Big Thing” ad campaign. In October of 2014, Magic Leap announced $542 million in funding from Google, Qualcomm, Legendary Pictures, and venture capitalists Andreessen Horowitz, Obvious Ventures and KPCB.
One of the best articles ever on Magic Leap appeared in the tech blog Gizmodo in November of 2014 titled “How Magic Leap is Secretly Creating A New Alternate Reality.” The article by leading tech writer Sean Hollister offers perhaps the best investigation into the Magic Leap technology to date with a brilliant survey of its patent applications and intellectual property. In the article, Hollister speculates (with a good amount of evidence to back him up) that a huge change happened for Magic Leap when Rony Abovitz ran into Eric Seibel from the University of Washington. As Hollister notes, for well over a decade, Professor Seibel had been developing an awesome new endoscope that uses a single optical fiber to safely see into tinier crevices in the human body than ever before. Using a piezoelectric to vibrate the tip of the fiber in a spiral pattern, it “scans” a much higher resolution image than you’d normally expect. It was just the sort of tool, Hollister notes, that a medical robotics company like Abovitz’s MAKO Surgical might consider using.
In spite of informative articles like Hollister’s, Magic Leap still remained largely an enigma to almost everyone. When the company did provide statements and press releases, there was a certain amount of ambiguity in them. Our product, the company said at this time, would be part of a “lightweight wearable” that transcends the concepts of virtual reality and augmented reality.
Abovitz’s LinkedIn Image
In October of 2014, Abovitz tells David Lidsky at Fast Company what his technology is not rather than what it is. “It’s not holography, it’s not stereoscopic 3-D,” he says. “You don’t need a giant robot to hold it over your head, you don’t need to be at home to use it. It’s not made from off-the-shelf parts. It’s not a cellphone in a View-Master.” Perhaps the best description of the Magic Leap technology came from the company’s press release that describes the technology by asking people to use their imagination. “Using our Dynamic Digitized Lightfield Signal, imagine being able to generate images indistinguishable from real objects and then being able to place those images seamlessly into the real world.”
2015 was filled with a number of press visitations to Magic Leap headquarters in Florida followed by some “Wow” articles. Not only members of the tech press were going down to Florida to see the hot new tech company but executives from most major media and tech companies were also making the pilgrimage to experience for themselves Magic Leap’s futuristic synthetic reality. But still, there was no product and the crowd outside waiting for the “show” to start began to get antsy and skeptical that they were being taken for a grand high tech ride. One person even posted the following online comment after one of the articles on Magic Leap: “Maybe its magic is raising over a billion dollars from investors?”
In Spring of 2015, the headline of an article in MIT Technology Review reads “Magic Leap Needs to Engineer a Miracle” noting that to make its prototype augmented-reality goggles a product, the company would have to scale up silicon photonics. It would not be an easy task, the article argued as heavyweights like Intel have struggled to do this.
By summer of 2015, news began to be increasingly dominated by the battle for the American presidency and subjects like revolutionary technology found it more difficult to rise above the growing volume of heated political rhetoric. Yet the company was still followed by the leading tech magazines and sites like the MIT Technology Review, Wired, Gizmoto and Fast Company.
* * *
Things had not changed as 2015 gave way to 2016. An article in a February issue of MIT Technology Review noted that Magic Leap has over $1 Billion in the bank and still no product. But the article noted that Magic Leap is spending plenty of money on other things like building a team that includes some notable names, such as science-fiction author Neal Stephenson, who’s the company’s chief futurist, and longtime video-game developer Graeme Devine, who’s its chief creative officer. Stephenson is an American writer and game designer known for his works of speculative fiction. His novels have been variously categorized as science fiction, historical fiction, cyberpunk and post-cyberpunk. His work explores subjects such as mathematics, cryptography, linguistics, philosophy and the history of science. He also writes non-fiction articles about technology in publications such as Wired. His breakout novel was the 1992 Snow Crash. Stephenson writes of his work with Magic Leap, “It is here that I am most likely to continue working on the sorts of transmedia projects that I have been interested in for many years.” The Magic Leap team is displayed on a company webpage with various toys representing team members. But click on the various toys and find behind them some of the most brilliant people in AI.
In April 2016, another article in the MIT Technology Review noted that Magic Leap has a headset but that its technology is still mysterious. Posted on YouTube in April of 2016 was a look around the Magic Leap office using the technology. Also, a tour of the offices given to a senior writer at Wired magazine.
In May of 2016, Magic Leap is the cover story in Wired magazine in a long, encompassing article written by legendary tech author and founder of Wired Kevin Kelly. The article is one of the best ever written on the current state of virtual reality. Kevin Kelly’s new book The Inevitable: Understanding the 12 Technological Forces That Will Shape Our Future is due out in June of 2016 and it seems a sure bet that mixed reality of Magic Leap will hold a prominent place in it.
* * *
So, here is the way things are, close to the middle of 2016, with the world’s hottest tech company. (This will all be such stale new so fast!) Again, much of what’s going on with it today has flown under the radar of popular knowledge due to the media concentration on the growing shouting match leading up to the Presidential election. But the scant information on the company is also due to the eccentrics of its founder Rony Abovitz as well as the inherent difficulty of describing something that has never existed before.
Even if the company lives up to that statement on its website that of making magic real for people, there are still skeptics who do not see some new utopian world when reality has become mixed and augmented reality. In March or 2015, an article in MIT Technology Review speculated that virtual reality advertisements will get in your face.
This is not hard to envision since the Internet has become wallpapered with product ads that follow you around like gum stuck to your cyber shoes. Visiting a site is like visiting a virtual retail store surrounded by things you’ve searched for on the Internet. The new scenario of mixed reality might lead to a consumer culture impossible to escape from with everything out there leading eventually to something to buy. Walking around the new world of AI might be like walking down the midway of one of Ray Bradbury’s dark carnival stories, the things we’ve searched for on the Internet yelling out at us like Carnival barkers.
Right now, the home page of the Drudge Report is bordered by ads for products one’s recently searched for online. The same thing when we appear on sites like Amazon to search for various products. At first, there were the product recommendations neatly laid out to see below our search. “If you like this product, we suggest that you might like this one.” In old sales terms it was called upselling or cross selling. Never let the sales prospect get away with just buying one product from the transaction. Suggest other links to him where he can chase products through cyberspace. But these formal recommendations once identified so clearly in writing through collaborative filtering technology have gone from being messages to mediums of messages.
* * *
Magic Leap’s Headquarters in Florida
The quote to Google shareholders from Google’s CEO Pinchai about devices fading away might be more prophetic and urgent than one might first think. For example, playing itself out right now are the woes of the greatest producer of technological devices in history. As noted in early May of 2016 by MarketWatch, Apple stock is down now and heading for its longest loss streak in 18 years. As one online reader commented, “Apple placed all its bets on its iPhone (device) and then was confronted with far better, faster, and cheaper models from an endless slew of competitors, and the result is that this one horse show is now getting driven out of town all over the world.” Do the woes of the great tech device maker Apple foreshadow the coming era when, as Sundar Pichai notes, devices will fade away in the new world of an invisible artificial intelligence? Are the troubles with Apple an early warning of troubles coming for all makers of technological devices?
The technology world has gone through a number of grand schisms. There was that first division between the Apple and DOS operating systems. The division between open source and closed source programming. But in recent years, the division has become more pronounced between the hardware of devices and the software inside devices or kept outside devices in data clouds.
No two companies represent the two sides of this division more than the device maker Apple and the software creator Google. Do the woes of the iPhone signify the beginning of devices fading away and new device less environments fading in?
Perhaps this is the case. Yet it seems that one of the continuing problems with Magic Leap is that it has been captive to a device itself. In order to move to the next step it will have to somehow lose those bulky, nerdy goggles one must wear for admission to their magic kingdom.
While the technology for mixed reality might be close at hand, it still needs a type of invisible interface with it. The goggles just will not do. It needs its own form of “mouse” device that has not been invented yet. Perhaps this will be in the form of some regular looking eyeglasses that are more fashionable than the nerdy looking Google Glass of a few years ago? Perhaps finding this invisible interface with the Magic Leap world is what’s holding back Rony Abovitz back from creating that first product of Magic Leap?
John Fraim has a BA in History from UCLA and JD from Loyola Law School. He is President of Midnight Oil Studios and GreatHouse Stories and President Emeritus of The Desert Screenwriters Group. He has been an entrepreneur all his life. Even while working for one of the world’s largest corporations in the world. He is a recognized authority on symbolism and author of the book Battle of Symbols as well as many articles on the Jung Site. He is author of Spirit Catcher: The Life & Art of John Coltrane. His articles have appeared in a wide number of publications from Psychological Perspectives, The Industry Standard, Adbusters, Business 2.0 and First Monday. His article “Electric Symbols” on the promises of Google appeared in the June 2002 issue of First Monday.
See his websites at: