From personal experience, there are not much difference in access time from China to servers hosted in Japan or United States. They are both slow as hell to access. (If there are any difference in access time it would be very small for any users to notice). The cause may be the fact that Chinese ISP's proxies all requests to foreign sites.
If your customers are indeed coming from China, the most logical place to host your server would be Hong Kong IMO. It has good access time from the Chinese mainland and from other Asian countries.
If you're serving China, is HK that much better? When I went there and talked to people they mentioned the need to account for the North/South telecom divide. It's essentially two countries in terms of data center infrastructure.
No, HK not noticeably better than Japan, and Japan is only a little better than the US.
You biggest problem is that everything going into mainland China goes through the Great Firewall, which tends to slow stuff down (both lag and latency). (Outgoing data is faster, which I suspect means they don't really monitor it as actively).
The only way around TGF is to host your site in a mainland datacenter, probably getting a .cn address, and * definitely* complying with China's internet regulations.
Google does this, but it can't serve results from a mainland center (as it doesn't want to follow mainland regulations), so it gives you a link to results served by google.hk (which I think is hosted in Hong Kong, but you can probably host a .hk address in the US or Japan, since HK is pretty lax with regulations).
If you want fast mainland connections, you need a mainland datacenter, probably a .cn address, and to do that you need to follow Chinese regulations.
I don't know about North/South. Possibly some stuff is done by provincial governments. Certain zones (like Shenzhen), and people (or companies) with permission might be able to get unfiltered internet. Maybe there's two Great Firewalls, and you need two centers to get inside both. I'm not an expert.
But I'm 99% sure that HK datacenters will have the same obstacles (re TGF) that non-Chinese datacenters have.
As itsnotvalid has pointed out, some sites are best going to the effort of getting a mainland address, simply because it means China will try to work out any issues with you rather than simply banning you. As the paperwork and features are likely to take a while (I'm not an expert, but paperwork always takes time, especially if it's in another language), you might want to start the process before they ban your main site, to avoid service interruptions (and maybe for goodwill - it can't hurt to look like you want to cooperate).
It depends of what kind of business you are dealing with. If you mainly serves mainland, then host in mainland as you don't want sudden ban of your site.
However, please note that hosting a site in mainland requires a registration at the respective ICP bereau hosting province.
