The Future of Google's Street View

The Need for Speed racing game franchise has been around for 16 years, sold over 100 million copies and generated more than $2.7B in revenue. People love to drive fast and they love to race, and it’s much safer to do this in a video game than in the carpool lane.

To make a racing game you need detailed models of the road and the nearby scenery. Some games use imaginary roads and locations created from scratch by designers. Other games painstakingly and expensively recreate real roads and real environments. The verisimilitude of seeing real landmarks whiz by adds to the fun.

In either case the cost of creating good quality content is one of the major costs of creating the game. In the future there will be third option. A game company will be able to license the entirety of the real world’s roads and all the surrounding areas from Google. The data will be streamed over the network from Google’s servers, where it is displayed seamlessly with the actual game elements.

The physical road surface will be mapped at centimeter resolution, every bump and rise accurately depicted. The visuals along the route will be in 3D and super high resolution, you could see the spots on a ladybug if you slowed down enough. And playback during the game will be silky smooth thanks to copious bandwidth and massive amounts of local hardware to warp the imagery in response to each subtle change in eye position.

Street View

Today Google’s Street View is an early preview of this virtual world technology. Google’s Street View was launched in May of 2007 and for the last 3.5 years Google’s cars and bicycles have been scouring the planet extending coverage, which is now comprehensive in many cities. Google recently announced that all 7 continents have at least some imagery: they added a few token panoramas in Antartica.

I have used Street View periodically. When I’m driving to a new location I sometimes use Street View to get my bearings and visualize landmarks. I’ve used Street View to see if a hotel was in a good neighborhood. I’ve used Street View just for nostalgia, to sneak a peek at places I used to live.

Despite my felicitous experiences with the feature, the sheer number of minutes I’ve used Street View is pretty small. I rely on Google Maps itself much more heavily, and use other Google products way more than Maps. So I have wondered, is it really worth the effort and expense of driving every mile of road of the planet just for this ability to peek at remote locations?

The investment starts to make sense when you think long term. Mostly what Google is doing is developing technology. From the camera clusters and range finders on the vehicles to server-side processing to client rendering. As the technology improves the quality of the virtual world will improve. Eventually the world will be useful for more than just “peeking at a remote location.” It will become a platform on which games and other graphically rich applications can be built. Google will likely license the content to 3rd parties, who will actually build the games and applications.

To make this all work Google will need to improve upon today’s Street View in 3 significant ways.

Removing all people and vehicles

Google blurs faces and license plates today. This is a processing step done after acquiring the imagery but before making it available for viewing. In the future Google will need to do even more sophisticated processing to remove all people and vehicles completely. Most games and applications will want to populate the world with their own computer generated vehicles and people, the static ones which happened to be there when the Google car drove by would be a distraction.

Removing these objects will require combining the data from multiple Street View runs, leveraging the fact that many cars and people will move between runs. However there will still be areas which are obscured in all runs. Google will employ techniques similar to Photoshop’s Content Aware Fill. This amazing feature infers what might have been under a deleted object, see for example this video.

Smooth movement

Today panning and zooming in Street View are smooth, but moving is more like slideshow, a blurry step from one location to another. By collecting tons more data and improving the client display algorithms it will eventually be possible to have completely smooth visuals during movement whether walking, driving or zooming along at cruise missile speeds.

Increased Resolution

Street View today is fairly high resolution in places but degrades quickly as you look far from the main path. Camera hardware is always improving, so that alone will improve quality over time.

The other way to improve resolution is to collect imagery from more locations, including those well away from any street. Google has reportedly been experimenting with quadcopters as a possible way to get additional vantage points. A swarm of Google copters could collect imagery far more comprensively than the Street View cars and bicycles.

Google may also be looking to acquiring imagery inside stores and buildings. Building interiors would certainly enhance the virtual world.

Once these technologies are in place and a smooth and believable virtual world exists, what sorts of games and applications will be developed? The already mentioned driving game is an obvious choice since Street View is focused on streets today, but as content for the wider world comes online, more possibilities open up.

Shooting Games

First person shooters are one the of the most popular game genres. You run around on foot or in specialized vehicles trying to shoot other game characters (AI or human controlled). As with driving games players crave new spaces to play in. The real world would provide an almost infinite variety of terrain and obstacles.

These games could benefit from the ability to inflict realistic damage on buildings and terrain. The idea of destroying and fighting around real world landmarks will be controversial, possibly even banned for some locations, but where permitted the target audience will happily embrace the real world as yet another play field.

Sim Games

Will Wright’s style of simulation games like Sim City and The Sims could be rewritten for Google’s virtual world. Initially games might keep the bulk of the world unchanged, just add it game entities on top. A more technically difficult version would allow changing the world itself, adding or subtracting buildings and even terrain, changing rivers and mountains. In either case you could always drop down into a first-person view, ride along with a delivery vehicle to see first hand how your new traffic plan is working out, for example.

Training and Monitoring

So called “serious games” will also be popular on Google’s platform. Bus drivers can learn their new routes, fire and police can practice cutting through traffic, disaster preparedness drills can be run with many online participants. In kind of a reverse simulation, the the position and status of real world objects, say all the police cars in a city, could be displayed in the virtual world for city officials to monitor. This is David Galernter’s Mirror Worlds idea from the early 1990’s.

Exercise Machines

It’s not a new idea to have a treadmill or exercise bike with a screen which shows some kind of virtual world, to motivate and entertain the exerciser. Hooking them up to Google’s virtual world would just be a way to vastly increase the amount and quality of available content. People could bike or run along scenic trails in Hawaii or Malibu or anywhere they choose.

Shopping and Social

Non-game activities like shopping or socializing in famous locations would be a draw for many people. In a virtual world you can interact seamlessly with people from all over the real world, wherever they are physically located. Similarly you can travel instantly, visit LA and Tokyo and Milan all in the same evening. This type of casual usage is reminiscent of the “metaverse” concept from Neil Stephenson’s Snow Crash.

Not all games will take place in Google’s virtual world. In fact most games will continue to build their own custom worlds, with fantasy or historical elements that just don’t exist in the real world. But a healthy subset of games and applications will, at least some of the time, reside in this new virtual world.

Google is not the only player in this game. Microsoft’s Streetside feature is a nearly identical to Street View. It is likely in the future there will always be handful of companies with the resources and motivation to model the entire world. These different companies will compete and will repeatedly re-image the planet as technology improves. The real world is a valuable source of content and from here on out it will be heavily and repeatedly mined for our benefit.

Postscript

Ten years after this post I received the below promotional email from Google Maps announcing the new gaming features of their Google Maps Platform! You just have to be patient!

• ←Previous
• Next→