Mobile app version of vmapp.org
Login or Join
Hamm4606531

: Best way to import 500MB csv file into mysql database? I have a 500MB csv file that needs to be imported into my mysql database. I've written a PHP script where I can upload the csv file

@Hamm4606531

Posted in: #Import #Mysql #Php

I have a 500MB csv file that needs to be imported into my mysql database.

I've written a PHP script where I can upload the csv file to and it analyses the fields and does the actual importing. but it can only handle small files of max. 5MB. So that's a 100 files and actually pretty slow(uploading)

Is there another way? I have to repeat this process every month because the data in the file changes every month it's about 12 000 000 lines.

10.02% popularity Vote Up Vote Down


Login to follow query

More posts by @Hamm4606531

2 Comments

Sorted by latest first Latest Oldest Best

 

@Nimeshi995

The best way to do this would be:


using a tool like [SQLyog] which has a free "community" version you can download here. You might find useful SQLyog FAQ about importing CSV data
import it [via SSH][4] (you can transfer database dump using an FTP and then just execute mysql -p -u username database_name < file.sql

10% popularity Vote Up Vote Down


 

@Sarah324

If you know that the file contains consistently-formed CSV (there is no standard, so you'll likely want to test extensively before importing over any production tables) you can skip PHP altogether and use MySQL's LOAD DATA INFILE statement after uploading the data to the server.

(If you don't have immediate access to the mysql console on the server, this could also be accomplished with a very simple PHP script or with phpMyAdmin)

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme