: Summary Minimising the attack vectors by not registering a domain, not registering hosting, not using a credit card, not openly giving out your IP and email address, not using Google Analytics,

Summary

Minimising the attack vectors by not registering a domain, not registering hosting, not using a credit card, not openly giving out your IP and email address, not using Google Analytics, and not blabbing about your new project will each reduce the chance of your identity being discovered.

How might your identity be discovered?

To understand how to set up a website anonymously, you must first appreciate that your identity might be obtained via:

The WHOIS record for any domains you register.
Your IP address (sent in the header of emails you send and found in server logs).
Your service providers' account records (i.e. hosting, domain registrar, ISP).
Your email address.
Your credit card details or other payment method.
Your Google Analytics account information (if tracking more than one site).

How can you eliminate most of these things?

The simplest and best way to eliminate many of these things being used to identify you is this:

Don't register a domain or pay for hosting at all.

Instead, register with a free blogging platform such as Tumblr or WordPress.com, who give you a subdomain when you sign up (i.e. yoursite.theirbrand.com). Many human rights bloggers and campaigners choose this option because the main thing they have to worry about then is protecting their email and IP address.

You can protect your IP address to a degree using a VPN such as HideMyAss whenever connecting to the Internet, or by using the Tor network. You should turn the VPN or Tor connection on before registering for any services and while using them.

A search for 'anonymous email' will provide you with an address you can use to sign up to services anonymously.

Be aware that some free blog providers will delete blogs that might be deemed offensive. Read their terms and conditions before signing up and take backups of your content if it's important to you.

If you feel compelled to register a domain and buy hosting

If, for some reason, you have to register a domain and buy hosting, consider the following ways of safeguarding your information:

Register the domain using an anonymous registration service (not the same as domain privacy), as mentioned in this question.
Use an anonymous hosting service that allows you to register under a pseudonym and without providing a postal address, such as Anonymous Speech.
Consider using hosting and registration services that support pre-paid methods that don't link your identity to the payment method (i.e. cash). As ever, be careful about sending cash by post.
Think about registering a domain in a country outside of your own nation's jurisdiction and hosting your site somewhere else too. This doesn't prevent requests for information being sent to hosting providers, but it can add an extra barrier.

Each of these items alone will not protect your identity from someone determined to discover it, but together they will make it harder to find out who you are.

Be careful about using Google Analytics and other third-party services

If you use Google Analytics -- even on a hosted platform like tumblr -- you should know that it's possible to do a 'reverse Google Analytics ID lookup' by putting your site's URL into services such as eWhois and Statsie to find out what other websites you're tracking with the same Google Analytics account. This could potentially unmask you if you're not protecting your identity on other websites you run.

For that reason, avoid using Google Analytics or any third-party service that might be used to identify you. Blog using a basic theme and no add-ons, widgets, or additional third-party scripts that require registration elsewhere.

Choose a unique nickname

If you're going to blog under an alias, make sure you've never used that alias for anything else. Aliases you've used in forums years ago ('runningWithScissors') have a habit of showing up in search engine results.

Don't go blabbing offline

Also consider not telling anyone else what you're doing. There's no point in taking steps to mask your identity online if you're willing to give it away freely offline.