WebGL and HTML5 Challenges For The Future

Sep. 11, 2011
comments

In the book Game Engine Gems 2: Chapter 13 author Rémi Arnaud examines different browser based 3d rendering techniques. According to Rémi, many 3D web rendering technologies failed because they could not get game developers on board. An example is VRML, which was pronounced dead on arrival by John Carmack.

Contents

#1 Browser: Fullscreen

Fullscreen defines a mode of viewing content over the entire screen. No window borders, decorations and so on are shown.

Why is it important?

Games for personal computers usually display fullscreen. Games often try to be an immersive experience without distractions such as address bars, scrollbars, operating system menus and bars etc.

Discussion

„I think fullscreen is crucial for making web games attractive.“
Mikael Emtinger (ROME, Technical Director)
WebKit has fullscreen support today. Chrome is adding it asap. I suspect Firefox is working on it as well.“
Gregg Tavares (Google I/O speaker, WebGL)
„We dont need last gpu techonology to make good game, but we can't make a game without Fullscreen and Mouse Capture“
Cedric Pinson (osg.js framework, WebGL services)

Current Status

Browsers support a sort of fullscreen mode, but this is often not true fullscreen. It may or may not cover the whole screen, and it usually leaves the address bar and some window decorations in place. It is also impossible for a html5 application to request the fullscreen mode.

How it can be fixed?

Browsers could offer an API that allows JavaScript to request fullscreen mode and have the user confirm it. Much like a user can allow websites to open popups.

What to observe

Often operating systems and their window managers force decorations into the user's face. The application will have to switch to an entirely different mode of display to get rid of it.

#2 Input: Mouse Capture

Mouse capture is a mode of interacting with the computer mouse that is not directly tied to a screen position of a cursor. It is used for a technique called mouse look. The operating system cursor is hidden and it is not possible to leave the application area by moving the mouse.

Why is it important?

Games have several choices to be presented to a user. A popular choice for games is the first person perspective where the mouse is used to control viewing direction. Games that do this usually capture the mouse, because:

  • Viewing direction has no relation to mouse screen coordinates, the mouse bumping against screen borders or exiting the viewing area has no meaning.
  • Seeing the cursor would be distracting

Discussion

„The various browsers are all experimenting with the best way to implement this and still prevent websites from -stealing- the mouse when you don’t want them too. Full screen support will also help this issue. See next question.“
Gregg Tavares (Google I/O speaker, WebGL)

Current status

Browsers do not allow JavaScript to capture the mouse input. The reason for this is that they want to prevent the user losing control of his mouse. If Mouse look is used anyway, there are problems when the mouse leaves the content area or bumps against a border of the screen.

How can it be fixed?

  • JavasScript could indicate its desire to capture the mouse to the browser
  • The browser could query to user to allow for capture by this application
  • The ESC key would be serving the function to escape the capture mode by default

What to observe

Mice often deliver higher quality input, which is then mapped to screen pixels. A naive implementation of mouse-capture resets the cursor position in the middle of the screen continuously. The genre of FPS games often attracts users who want a high sensitivity for their mouse. Clamping mouse coordinates to pixels leads to jerky and unsmooth mouse look behavior.

#3 WebGL: Multiple Render Targets

Multiple Render Targets is the ability to store rendering output in multiple textures at once. The capability can be queried in OpenGL by the value of the gl_MaxDrawBuffers. The WebGL 1.0 specification allows for the use of multiple render targets in GLSL by the use of gl_FragData.

Multiple render targets are supported since Direct3D 9.0 and OpenGL 2.0 core.

Why is it important?

Techniques like Deferred Shading and Screen Space Ambient Occlusion rely on data produced during a render pass such as per-fragment colors, normals and positions per fragment.

If multiple render targets are not supported, a scene has to be rendered multiple times (passes) to capture the required information, which can be quite expensive.

Discussion

„If and when OpenGL ES 3.0 comes out, if it supports multiple render targets then maybe WebGL 2.0 will support them. If you feel strongly about this bring it up on the Public WebGL mailing list.“
Gregg Tavares (Google I/O speaker, WebGL)
„Rendering in multiple passes can sometimes make things very slow.“
Mikael Emtinger (ROME, Technical Director)

Current status

Since the WebGL 1.0 specification and implementations lacks a way to specify which target index a texture is attached to, multiple render targets cannot be used.

How can it be fixed?

  • Khronos could revise the WebGL specification to allow multiple render targets
  • Google could update their ANGLE wrapper to allow for multiple render targets for anybody with OpenGL 2.0 or Direct3D 9.0.

#4 WebGL: Geometry Instancing

Geometry instancing is the ability to draw the same object multiple times that has some instance specific data with a single draw call. The OpenGL extension Draw Instanced introduced in 2008 supports this feature.

Draw instanced is supported since OpenGL 2.0 and Direct3D 9.0

Why is it important?

Games often draw the same object many times for each frame, such as vegetation, particles, avatars, physical rigid bodies, etc. Without a draw instanced call, this results in a lot of draw calls or massive buffer updates (pseudo instancing) which usually affects performance quite negatively.

Both mobiles and desktops would benefit from this feature. The alternative to instancing is doing a lot of javascript computation and data updates to the GPU or a lot of individual draw calls.

Discussion

„If the feature can be emulated relatively easily on hardware that does not support it there is a higher possibility it will be added. If you’d like to see it added you can lobby for it on the Public WebGL mailing list.
Gregg Tavares (Google I/O speaker, WebGL)

Current status

The WebGL 1.0 specification contains no API to use instanced drawing.

How can it be fixed?

  • Khronos could revise the WebGL specification to allow for draw instanced
  • Google could update their ANGLE wrapper to allow for draw instanced for anybody with OpenGL 2.0 or Direct3D 9.0.

#5 WebGL: Texture lookup in the Vertex Shader

Programmable hardware rendering uses two stages to produce a picture: The vertex and fragment shader. Traditionally only the fragment shader could sample from textures.

Texture lookup in the vertex shader are supported since Direct3D 9.0 and OpenGL 2.0 core.

Why is it important?

It can be used for techniques such as displacement mapping, physically waving grass and particle effects.

Discussion

„The ANGLE project is in the process of adding support for Vertex Shader texture lookups.“
Gregg Tavares (Google I/O speaker, WebGL)

Current status

An updated version of the ANGLE library for windows (r686) supports this feature.

  • Firefox 6 uses this version
  • Google Chrome 12 does not yet use this version

How can it be fixed?

Google could update Chrome to use a newer version of their ANGLE library.

#6 WebGL: Floating point textures

Traditionally textures are limited to one byte per channel. That allows for only 256 different states per channel. Floating point textures use 4 bytes per channel, which is enough for a IEEE floating point value.

Floating point textures are supported since Direct3D 9.0 and OpenGL 2.0 core.

Why is it important?

Techniques such as High dynamic range rendering, Deferred Shading, Screen Space Ambient Occlusion and Shadow Mapping greatly benefit from the added accuracy. The lack of floating point textures can lead to artifacts or make a technique infeasible altogether.

Discussion

„Request support on the ANGLE project and see what they say.“
Gregg Tavares (Google I/O speaker, WebGL)

Current status

An updated version of the ANGLE library for windows (r686) supports this feature.

  • Firefox 6 uses this version
  • Google Chrome 12 does not yet use this version

How can it be fixed?

Google could update Chrome to use a newer version of their ANGLE library.

#7 Browser: Choice of native OpenGL and ANGLE Direct3D for Windows users

OpenGL is an API standard that is well supported by the three major vendors of graphics hardware (ATI, Nvidia and Intel). Direct3D is a competing standard by Microsoft. The ANGLE wrapper by Google translates OpenGL API calls to Direct3D 9.0.

„The goal of ANGLE is to allow Windows users to seamlessly run WebGL content by translating OpenGL ES 2.0 API calls to DirectX 9 API calls.“
ANGLE Project

Why is it important?

There is some truth to this notion that 3D hardware drivers are notoriously unreliable or absent.

  • The same is true for Direct3D which is often outdated, buggy or not available.
  • OpenGL drivers for non Nvidia hardware are usually not as good.
  • A lot of WebGL will run just fine with native OpenGL but fail to work on Direct3D, even if no features are used that ANGLE(Direct3D) does not like.
  • People who've bought their hardware for gaming or other 3d related tasks, often have a surprisingly good set of hardware and drivers.
  • Given an up to date version of OpenGL drivers and Direct3D, OpenGL often works better and faster.

Discussion

„No, it's too complex of a question for a normal user. Maybe it could be more visible in the options, but what would be really cool would be for the developer, through JS, to be able to choose preferred implementation.“
Mikael Emtinger (ROME, Technical Director)
„OpenGL works for AAA games. OpenGL-support was never an issue.“
Mikkel Gjol (Graphics Programmer at splash dammage) on Brink (AAA video game)

Current status

By default Chrome and Firefox use ANGLE/Direct3D on Windows. Although this setting can be changed by a user, it's difficult and inconvenient to do.

Issues

  • Firefox uses Direct2D and benefits from fast compositing with Direct3D. Enabling native GL degrades firefox page redraw performance.

How can it be fixed right now?

  • Browsers could make a decision as to what would be the better initial driver (OpenGL or Direct3D) instead of defaulting to Direct3D.
  • WebGL applications could indicate which driver they think will work better.
  • Users could be given a global and per-site specific choice to override whatever the Browser/WebGL app figured out would be best.

How should it be fixed eventually?

  • PC vendors should see to it that their customers get up to date drivers initially, and continually by updates.
  • Graphics hardware vendors should provide good drivers for their hardware