PDA

View Full Version : What pathname to run LOAD DATA?


bdunning
6-20-05, 12:46 PM
I have a mega CSV file that I need to upload to my server to run LOAD DATA with (using the SQL command in phpMyAdmin). So far, all the combinations of pathnames I've tried are not working.

1. To where should I upload the file?

2. What pathname do I use in my LOAD DATA statement?

(I can't simply use phpMyAdmin's text file upload, since my file is several hundred megs in size.)

RTH10260
6-21-05, 12:05 PM
I have a mega CSV file that I need to upload to my server to run LOAD DATA with (using the SQL command in phpMyAdmin). So far, all the combinations of pathnames I've tried are not working.

1. To where should I upload the file?

2. What pathname do I use in my LOAD DATA statement?

(I can't simply use phpMyAdmin's text file upload, since my file is several hundred megs in size.)First you will need to prepare such a large file, cause you have a query limit that will not permit such a large chunk to be loaded in one.

- Split the file into cunks of not more than 72'000 rows.
- If loading goes nice, you will want to define to additional usernames for a roundrobin access within a single hour (each username has above hourly limit).
- you may need to further reduce the chunk sizes so that you manage thru with cpu limits and connection timeouts (if you run the script interactive). Trial and error approach to find the correct settings.
- Upload location is your choice within your account, you can create a directory at level of the /htdocs to keep the files out of the reach of the webserver.
- The path for your script will be '/www/u/user/newdirectory/whatever' (replace 'u' and 'user' with the name of your account path).

If you have a correct working command for mysqlimport , you could possibly ask support to run this for you using the complete file, the file already uploaded in your account space. I have seen that Support does this extra service for large MySql import jobs.