castAR is a projected augmented reality system that displays holographic-like 3D projections right in front of you. Star Wars: Episode IV – A New Hope sparked our imaginations of what might be possible in the future by showing R2-D2 and Chewbacca playing a holographic 3D chess-style game. It has taken nearly 35 years since the film was released for this technological dream to come to reality, but with castAR, that reality can be yours.
One primary concern we kept in mind when developing the castAR system was how to bridge the gap between the physical world and the virtual world. We accomplished this through the use of two unique peripherals: the Magic Wand and the RFID Tracking Grid. The Magic Wand is new kind of controller that lets you position things in space and control them with a joystick, all with one hand. The RFID Tracking Grid allows you to uniquely identify, track, and augment physical objects (such as miniatures, cards, and board game pieces) across the surface.
Another goal we have is to make castAR a comfortable and enjoyable experience for a wide audience. The final retail glasses are expected to weigh less than 100 grams, which is only slightly heavier than a pair of sunglasses. If you wear prescription glasses, castAR was designed to sit easily on the outside of them.
Finally, we also strive to make castAR the most versatile head-mounted display available. We offer an attachment that transforms castAR into a true virtual reality system as well as a true augmented reality system. Whichever reality you prefer, there is now a single system capable of taking you there.
castAR’s projected augmented reality system is comprised of two main components: a pair of glasses and a surface. The frames of the glasses contain two micro-projectors—one for each eye. Each projector casts a perspective view of a stereoscopic 3D image onto the surface. Your eyes focus on this projected image at a very natural and comfortable viewing distance. A tiny camera in-between the projectors scans for infrared identification markers placed on the surface. The camera uses these markers to precisely track your head position and orientation in the physical world, enabling the software to accurately adjust how the holographic scene should appear to you. The glasses get their video signal through an HDMI connection. The camera is connected via a USB port on the PC. We are still experimenting with communication options on mobile devices.
The surface is made of retro-reflective sheeting material, similar to the kind used in traffic signs and high-visibility safety clothing. The primary benefit to using this material is that it bounces the majority of light from our projectors directly back toward the glasses with very little scattering. This enables the simultaneous use of a single surface by multiple people while keeping each viewer’s view private from the others.
Since your vision is focused at a natural viewing distance, you shouldn’t experience eye strain. Projected augmented reality allows you to simultaneously see both virtual and real-world surroundings, so you are spared other sorts of discomfort as well. For example, an important aspect of your body’s understanding of the physical world is tied to your inner ear—the part of your body responsible for balance and motion sensing. When you are able to see your physical world, your eyesight and inner ear will stay in sync with your movements. Most people do not feel nausea or motion sickness when using castAR and projected augmented reality.
- castAR Software Development Kit: If you are doing a custom game or other application, you can use the SDK to get access to tracking data / rendering matrixes as well as RFID data.
- Unity integration: Simply create an empty GameObject and add our interface script to it. You’ll be off and running under castAR within a few minutes. Additional scripts for the Magic Wand and RFID objects are also available.
- Demos and example experiences: We will provide you with demo games and other sample projects to help give you an idea of a few things that are possible with this system.
- (Potential) Additional engine support: We are currently in discussions with other engine developers to add support for castAR.
- (Potential) Support for existing VR applications: We are also investigating ways to get existing VR applications up and running on our system.
- GlassesThe castAR glasses contain a set of two high resolution micro-projectors. Each projector independently refreshes at 120 hertz and has a very high fill factor (i.e. no screen door effect). While the retro-reflective surface eliminates the majority of the cross talk between projectors, active shutters eliminate the rest. There is also an integrated tracking camera (see the Tracking Camera section below). They easily fit over prescription glasses; final weight is expected to be less than 100 grams.
- AR & VR Clip-OnThis clip-on attaches to the front of the glasses to transform your experience into either true AR or true VR. True AR allows you to use our glasses without the retro-reflective surface, augmenting the real world. True VR is a fully synthesized environment; the computer generates all aspects of the visuals you see. The clip-on has been designed to be comfortable and lightweight, yet still provide the immersive atmosphere you want. With this component, you will have no need for any other head mounted display.
- Tracking CameraThis highly accurate tracking system is capable of detecting movements of sub-millimeter. It is also extremely fast, capable of tracking at 120 hertz, and is low latency, cutting out several frames of delay compared to other tracking schemes. Custom hardware logic on the glasses handles localized processing of the image data. As this logic does all of the analysis-intensive work, the connected computer simply receives the results, thus freeing its CPU up for other tasks. The system is also very power efficient and will have negligible impact on your battery life when used with mobile devices. The tracking camera can be purchased separately to be used on head-mounted displays that don’t have highly accurate tracking, such as those which rely solely on gyros.
- Magic WandThe Magic Wand can be used as both a joystick and a 3D input device. The handheld portion of the wand has buttons, a trigger, and a thumb stick. The wand also tracks its own movement in 3D space, allowing you to seamlessly interact with your virtual environments.
- RFID Tracking GridThe RFID Tracking Grid sits underneath your surface. When used with the bases (see the Bases section below) or your own RFID tags, the software can track and uniquely identify them across the surface. This allows you to associate tags with physical objects such as miniatures, cards, board game pieces, or anything else that fits on top of the surface. You can use the tracking and identifying abilities to augment your pieces, such as displaying stats or a health bar next to a physical miniature, or calculating the distance between two separate miniatures.
- BasesBases are designed to allow you to conveniently use your existing miniatures. For most standard-sized miniatures, our base will easily snap onto the bottom. Bases come in two varieties: RFID and RFID Precision. RFID allows you coarsely track and uniquely identify game pieces across the RFID Tracking Grid. Precision Bases have a custom circuit board that allows for both RFID tracking as well as two-way communication. The two-way communication can be used for such things as increasing the precision of the position tracking or to control small motors and other miniature electronics. For example, you could have a dragon miniature with a Precision Base that has a tiny smoke generator and some red LEDs attached to it. The software could then tell the base to turn on the smoke generator and the red LEDs, providing an illusion that the dragon is breathing fire.
- Larger SurfaceThis surface is twice the size of the standard one, for people who want a larger area to experience their projected augmented reality on.
We’ve designed castAR to be a very integrated experience with your physical world. To accomplish this, we’ve created many compelling options. In order to easily add on some of the accessories that might go with your main pledge reward, we’ve created a handy reward calculator to help you pick your options and determine your final pledge amount. The following options, described in detail in the previous section, are available:
- Large Surface: This 1 meter by 2 meter surface is perfect for even the largest tables. Expansive gameplay has never been so… expansive!
- RFID Tracking Grid: The RFID Tracking Grid, when used with the RFID Bases, RFID Precision Bases, or your own RFID tags, can track and identify the tags across the surface. The included tracking camera, when used with Precision Bases, can provide extremely accurate positional data. The RFID Tracking Grid is perfect for board gaming, war gaming, or other uses that require physical objects and identifying aspects about them.
- RFID Base 10 Pack: These bases contain a unique RFID tag and are designed to easily snap onto the bottom of most standard sized miniature bases. Use these with the RFID Tracking Grid to coarsely track them across the surface and uniquely identify them in software.
- RFID Precision Base 10 Pack: These bases have a small custom circuit board that allows for two-way communication between the board and the RFID Tracking Grid. The circuit board contains a programmable micro-controller. By hooking up the included IR LED stalk, the tracking camera on the RFID Tracking Grid can precisely determine positional data. They are pre-programmed for precision tracking by default, but experienced programmers can code them for functions, such as turning on small motors, enabling tiny smoke generators, flashing LEDs, etc. Imagine having software drive a tiny car with motors around a virtual race track.
- Magic Wand: Slicing and dicing; stabbing and poking; sometimes you just need a second wand to crush your enemies!
- AR & VR Clip-On: Why settle for one reality? This clip-on will give you the ability to transform castAR into a true VR as well as a true AR system.
- Tracking Package: Do you already own a head mount display? Consider adding this option to bring the accurate and fast tracking of castAR to it! This package includes a tracking camera and markers.
It began with one of those Eureka moments—around May of 2012, Jeri was (as usual) working in Valve Software’s hardware lab late at night. She was doing some experiments with a projector and saw an unexpected flash of light on the opposite side of the room. She observed that when the projector shone onto a particular surface, it produced an extremely bright flash as light reflected back. Investigating the cause of this reflection, she discovered it was a piece of retro-reflective material. Jeri quickly realized that this material could be used as a surface for projecting images onto. It was so efficient at bouncing light back, you could use small micro-projectors that emit tiny amounts of light and still see a very bright image.
The first prototype Jeri created was called the “head crab” due to its large size and the pain that wearing the heavy model caused. It had two cellphone sized projectors and was only capable of displaying a static image without tracking. However, this was enough to demonstrate that the concept could be used as an AR device, so she quickly started to build the next prototype.
Rick had been working at Valve Software for nearly four years when he and Jeri had independently gone to California Extreme (an annual pinball and arcade show) in August 2012. During some downtime, Jeri told Rick about her AR prototype project, lamenting the fact that she couldn’t find anyone within Valve to help her with the software side. Rick, already dedicated to the small team optimizing Linux ports of Left 4 Dead 2 and Team Fortress 2, decided to devote his spare time outside of work assisting Jeri. He spent the next 5 months developing a prototyping system that could easily create and test AR experiences.
In early 2013, an ‘opportunity presented itself’ so that Jeri and Rick were able to acquire the technology, forming a new business together to continue the project independently. Over the course of the next 8 months, they spent an average of 14 hour days, 5-6 days a week, pouring most of their energy, lives, and personal savings into expanding and creating new technologies that have become castAR.
Anticipating the need for crowdfunding to raise capital for bringing the final product to market, they decided that the best way to make people believe in the project was to let them experience it firsthand. In May 2013, castAR was unveiled at the Maker Faire Bay Area. The show was a huge success, generating curiosity and excitement. Lines to get into the Technical Illusions booth were at least an hour long each day.
In the last 6 months, we have since shown castAR at more than half a dozen trade shows and conventions while continually refining and improving the hardware and software. We have been exploring more ways to bring the physical world into the virtual world with a wider range of experiences and demos, and hope to inspire others’ imaginations for what the future could bring.
Jeri Ellsworth is an inventor, product designer, and engineer of both chip and system-level designs. Her broad skillset enables her to design reliable, low cost, and highly integrated systems, and has made her well-known in the industry for cutting-edge consumer products which sell in the millions.
She often holds the lead designer position on consumer products and has driven many projects from concept to mass production, including electrical and mechanical prototypes, cost reductions, certifications, tooling, tests, and overseas mass production.
Outside of engineering, Jeri has been a key member in several startups and has built businesses from the ground up. She is intimately familiar with day-to-day operations as well as how to build efficient engineering teams in new company environments.
Jeri has become an icon in the maker / educational community for pushing the boundaries of what was thought possible for an individual. She offers her time as a mentor and has produced hundreds of instructional videos demonstrating complex science subjects in an approachable way free of charge. She is a frequent lecturer at universities and speaks at many annual events about creativity and engineering. In 2012, she was presented the Maker Hero of the Year award by MAKE Magazine.
Rick Johnson started working on his first professional video game, Black Crypt, half way through college with 3 other people. Raven Software was formed during the game’s development. The game was published by Electronic Arts on the Commodore Amiga. After graduating college, Rick went to Raven full time, working on over 13 titles includingHeretic 2 and Star Wars Jedi Academy. He was the lead programmer on Hexen 2,Soldier of Fortune, Soldier of Fortune 2, and Quake 4. After 17 years at Raven, Rick joined Gearbox Software to work on an unannounced prototype project and to assist withBrothers in Arms.
He then moved on to Valve Software. There, he improved editor tools, created the majority of the graphics technology in Dota 2, and was one of three primary founders of the Linux cabal. His contributions helped make Left 4 Dead 2 run on OpenGL in Linux just as fast as its Windows counterpart, helping bring Linux gaming into the modern age.
During his last 5 months at Valve, he spent most of his spare time outside of work helping create a prototyping environment for the AR project that Jeri Ellsworth was involved in. Together, they formed Technical Illusions and continued to iterate on and improve the hardware and software associated with castAR.
Taking functional prototypes all the way to quality retail products is a complex process that requires additional resources, both in capital and manpower. Now that we are at the point where we have proven our various technologies and demonstrated them in public, we have come to Kickstarter to help provide us with the assistance we need to become market-ready.
- switching development practices towards production practices, such as going from using an FPGA that allows for fast hardware development iteration to an ASIC, which is the last step of producing the final chip that drastically reduces power consumption
- software support to complete the SDK and engine integrations; creation of experiences, games, and samples for the public
Fortunately, the experience we have shipping products in the past allows us to know and be able to plan the steps we need to take at this point. The following lists break down the high level type of work and resources that are needed to bring castAR to market:
- Engineering – mechanical, industrial, and ergonomics design, PCB and ASIC development, thermal and drop analysis, and accelerated failure testing
- Manufacturing – line setup fees, per minute assembly, per minute test, ROM parts, printing costs, packaging
We hope that by providing you with this detailed level of information, you can see some of the expertise we’ll need to call upon in successfully driving this product toward completion. Our policy has always been to be upfront with people, to demonstrate castAR in public, and let people try it for themselves. By conducting our development in such a transparent manner, we hope to gain your confidence and trust.
Over the next month, we plan on providing you with new video footage and more in-depth details about castAR. Some of the videos we are planning include a demonstration on how to hook a Unity game up to castAR, detailed talks about how the technology works, and focused explorations of some of the different experiences and games we are currently working on.
Please visit our website and forums at http://www.technicalillusions.com and feel free to ask us questions or share your ideas on what you would like to see more of during the campaign. We would love to hear from you!
Delayed shipment — Developing a product is a complex combination of engineering tasks, working within a manufacturer’s capacity, coordinating the availability of parts, and making sure they get delivered on time to the manufacturer. We’ve solved the difficult engineering problems (which we’ve demonstrated publicly with our prototypes) and our team is already familiar with overseas manufacturing, so the risk of delay is minimal.
Delay due to end-of-life parts or single source parts becoming suddenly unavailable — When possible, we will opt to have multiple vendors supplying parts so substitutions can be made with minimal impact to the design cycle.
Delay due to world issues — Natural disasters, fires at factories, or governmental policies can occasionally cause delays in manufacturing and logistics. If any of these should occur, we will communicate with you throughout the entire process so nothing will come as a surprise.
Does not function as advertised — We’ve demonstrated all of our technology to thousands of people in live events over the last 6 months. We’ve also made special efforts to only use castAR technology in our videos and represent the product as you will receive it.