Drupal's files directory can get enormous, and it's best not to keep it in version control. Here's a way to use Apache's proxy module to point your local environment directly to your dev/live site's files directory so you don't have to download it.
In your httpd.conf:
- ProxyRequests On
- ProxyVia On
- <Proxy *>
- Order deny,allow
- Deny from all
- Allow from localhost
- Allow from all
- <Location /sites/default/files/imagecache/>
- ProxyPass http://dev.yoursite.com/sites/default/files/imagecache/
Of course you'll want to tailor it to your needs; maybe you want only one imagecache preset proxied, or just one directory specifically on one local host. Either way, this recipe will get you started. Hope it helps!