2018-12-10

HW5 Troubleshooting.

I'd like to quickly post a few fixes to problems I encountered throughout HW5, while still making an effort to avoid doing anyone's homework for them. These are issues that aren't well documented and ended up giving me a headache for an hour trying to debug, largely due to changes in version of MySQL or simply because people assume certain things are common knowledge.
mysqlimport --secure-file-priv
One problem that someone else in the class and I ran into was an error saying something along the lines that "--secure-file-priv" was preventing the mysqlimport line from executing. This is a value that belongs to the server that is hosting your SQL database. The internet states in several places that a solution to this is using the "--local" option when running mysqlimport, but I only got other errors from this and ultimately something different worked for me:
If you installed MySQL through the MySQL website and did the whole installer process, you hopefully have a program on your computer called MySQL Workbench. If you open this and access your database, on the left there's a button in the navigation called Options File. Click this and then the Security tab, scroll down and there should be the --secure-file-priv setting. Do not uncheck the box, just delete any text in the box to the right and then hit Apply.
Resource Description for 01_secure-file-priv.png
If you didn't install MySQL through the website, you should have a my.ini file that has a --secure-file-priv line somewhere in it. Set it's value to "", which is just an empty string and should be equivalent to what I did through the Workbench.
mysqlimport options
If you created a comma-delimited .csv file for MovieExec and Studio like I did (I did mine through Excel), you may also get an error after --secure-file-priv that says the csv couldn't be read properly due to an error on line 1. Here are a few handy options you can use that fixed a series of weird problems I had:
  --fields-terminated-by=,
Putting this in your mysqlimport line will properly split each line by commas. I believe the default is a tab, or \t, so you will probably need this option.
  --lines-terminated-by=\r\n
You can probably guess that this splits each row by a carriage return \r and a new line \n. Some people may not have this same problem that I did, but my csv had a \r\n after every line instead of a \n. I think just a \n is default, so I also needed this option.
  --ignore-lines=1
This line makes mysqlimport ignore the first entire line of the csv file. At first I didn't see any reason to use this because I didn't add headers to my csv file. But then I saw an unusual problem where the very first row that mysqlimport brought in would have a question mark at the front, and I think this problem was caused by my \r\n in the lines-terminated-by option. So to fix that I just added column headers to my csv files through Excel and then used this ignore-lines option.
path and classpath
There are about 100 ways of doing this and I think anyone who has used Java for a class in the past knows that this problem is oddly frustrating. Or maybe I'm the only one. Every time I want to execute the javac command I have to first specify my Java \bin folder, which for some reason has never worked properly through the Windows Environment Variables settings. If you encounter a problem like "javac is not a recognized program", try this:
  set path=%path%;C:\Program Files\Java\jdk1.8.0_161\bin
Of course, I probably don't have the same version of Java as you, so please check your Program Files folder and change the jdk1.8.0_161 to whatever your actual version of Java is.
Beyond that if you also get a "could not find the drivers specified" problem for JDBC, I fixed that in a similar way:
  set classpath=%classpath%;your-path-here\mysql-connector-java-8.0.13.jar;
But again please set the proper path to the connector, and make sure you have the same version connector as me.

Those are all the main issues I faced. Feel free to use this thread to post your own troubleshooting problems and solutions.
(Edited: 2018-12-10)
I'd like to quickly post a few fixes to problems I encountered throughout HW5, while still making an effort to avoid doing anyone's homework for them. These are issues that aren't well documented and ended up giving me a headache for an hour trying to debug, largely due to changes in version of MySQL or simply because people assume certain things are common knowledge. '''mysqlimport --secure-file-priv''' One problem that someone else in the class and I ran into was an error saying something along the lines that "--secure-file-priv" was preventing the mysqlimport line from executing. This is a value that belongs to the server that is hosting your SQL database. The internet states in several places that a solution to this is using the "--local" option when running mysqlimport, but I only got other errors from this and ultimately something different worked for me: If you installed MySQL through the MySQL website and did the whole installer process, you hopefully have a program on your computer called MySQL Workbench. If you open this and access your database, on the left there's a button in the navigation called Options File. Click this and then the Security tab, scroll down and there should be the --secure-file-priv setting. Do not uncheck the box, just delete any text in the box to the right and then hit Apply. ((resource:01_secure-file-priv.png|Resource Description for 01_secure-file-priv.png)) If you didn't install MySQL through the website, you should have a my.ini file that has a --secure-file-priv line somewhere in it. Set it's value to "", which is just an empty string and should be equivalent to what I did through the Workbench. '''mysqlimport options''' If you created a comma-delimited .csv file for MovieExec and Studio like I did (I did mine through Excel), you may also get an error after --secure-file-priv that says the csv couldn't be read properly due to an error on line 1. Here are a few handy options you can use that fixed a series of weird problems I had: --fields-terminated-by=, Putting this in your mysqlimport line will properly split each line by commas. I believe the default is a tab, or \t, so you will probably need this option. --lines-terminated-by=\r\n You can probably guess that this splits each row by a carriage return \r and a new line \n. Some people may not have this same problem that I did, but my csv had a \r\n after every line instead of a \n. I think just a \n is default, so I also needed this option. --ignore-lines=1 This line makes mysqlimport ignore the first entire line of the csv file. At first I didn't see any reason to use this because I didn't add headers to my csv file. But then I saw an unusual problem where the very first row that mysqlimport brought in would have a question mark at the front, and I think this problem was caused by my \r\n in the lines-terminated-by option. So to fix that I just added column headers to my csv files through Excel and then used this ignore-lines option. '''path and classpath''' There are about 100 ways of doing this and I think anyone who has used Java for a class in the past knows that this problem is oddly frustrating. Or maybe I'm the only one. Every time I want to execute the javac command I have to first specify my Java \bin folder, which for some reason has never worked properly through the Windows Environment Variables settings. If you encounter a problem like "javac is not a recognized program", try this: set path=%path%;C:\Program Files\Java\jdk1.8.0_161\bin Of course, I probably don't have the same version of Java as you, so please check your Program Files folder and change the jdk1.8.0_161 to whatever your actual version of Java is. Beyond that if you also get a "could not find the drivers specified" problem for JDBC, I fixed that in a similar way: set classpath=%classpath%;your-path-here\mysql-connector-java-8.0.13.jar; But again please set the proper path to the connector, and make sure you have the same version connector as me. ---- Those are all the main issues I faced. Feel free to use this thread to post your own troubleshooting problems and solutions.

-- HW5 Troubleshooting
    Thanks David! The error I've been getting was from not putting in lines-terminated-by.          
    This helped!
Thanks David! The error I've been getting was from not putting in lines-terminated-by. This helped!
X