Confused about the right path to dataset in my code [Solved]

[fishkeeki]

Hey I am new to floydhub and am still struggling with putting the right path to locate the data in my code.

I read the documentation on mounting data and uploaded my dataset accordingly.

After issuing the commands below to run the script

floyd run --gpu --env pytorch-0.4 -m "1st run" --data yubin627/datasets/deepfashion_watchout/1:raw_deepfashion_dataset --data yubin627/datasets/deepfashion_watchout/4:models "python train.py"

I got this error message:
link to my job page
link to the dataset

FileNotFoundError: [Errno 2] No such file or directory: '/raw_deepfashion_dataset/Eval/list_eval_partition.txt'

checking the "Input Data" tab under my job page, I still don't find any problems with this path in the error message "/raw_deepfashion_dataset/Eval/list_eval_partition.txt". The file does exist - need another pair of eyes to kindly help me figure out what's going on here. Thank you!!

[redeipirati]

Hi @fishkeeki,

checking the "Input Data" tab under my job page, I still don't find any problems with this path in the error message "/raw_deepfashion_dataset/Eval/list_eval_partition.txt". The file does exist - need another pair of eyes to kindly help me figure out what's going on here

From the mounting point, you have another raw_deepfashion_dataset folder to traverse. Here's the right path:

'/raw_deepfashion_dataset/raw_deepfashion_dataset/Eval/list_eval_partition.txt'

My advice is to change the mounting point name of the first dataset to avoid confusion such as floyd run ... --data yubin627/datasets/deepfashion_watchout/1:input ... and then use this path to get access to the file of interest:

'/input/raw_deepfashion_dataset/Eval/list_eval_partition.txt'

Hope that helps.

[fishkeeki]

Thank you very much for the quick reply and the advice!
Yes it is working now!

I think I got myself confused on the mount point name - I thought it must be the name of the data folder I uploaded; if I understand it correctly, actually it is just like creating a directory - the name of the mount point could be anything.

so for some data that i uploaded to username/datasets/datasetname/1, I can mount it to my project with a command like floyd run --data username/datasets/datasetname/1:anything "python train.py". In my script I can reference the data (say, train.csv) by "/anything/train.csv"

[redeipirati]

if I understand it correctly, actually it is just like creating a directory - the name of the mount point could be anything.

Almost anything. You cannot specify as mounting point one of the top-level folder of the FHS (such as /etc).

so if I understand it correctly now, for some data that i uploaded to username/datasets/datasetname/1, I can mount it to my project with a command like floyd run --data username/datasets/datasetname/1:anything "python train.py". In my script I can reference the data (say, train.csv) by "/anything/train.csv"

Yes, correct.