In Azure you will need to add a line to your web.config to allow the Umbraco 404 error page to work, otherwise your users will be greeted with the following message:

The resource you are looking for has been removed, had its name changed, or is temporarily unavailable.
The change you need to make in your web.config is as follows:

<configuration>
...
<system.webServer>
<httpErrors existingResponse="PassThrough" />
...
</system.webServer>
...
</configuration>