Downloading a dataset

[shahidusmanm]

Hi!

How do I download datasets from the internet?
I wrote a custom script to download my data from a website but can't understand how to run it. I did try running it and the system showed it downloaded all the data but I can't seem to find it anywhere on my account.

The Floyd Documentation on Downloading Custom Datasets is unavailable (I get a 404).

TIA!

[ririau]

A whole bunch of the docs are 404, but here's what you may have been after: http://docs.floydhub.com/commands/data/

Often I find I need to naviage to docs.floydhub.com and then navigate to the relevant page.

I was able to upload datasets using this guide, it's not too hard:

• Create a dataset on your floydhub homepage (click "datasets" in the top nav). Remember the name you gave it.
• Download the data to your local machine, in its own directory
• Run the dataset init command on your machine in that directory
• Run the dataset upload command
• It should now be available for you to mount as a dataset

[shahidusmanm]

Hey Ririau!

Thank you for your reply.

But unfortunately, that's not what I'm after. See my dataset is large (about 2+GB) and uploading that over my current internet connection is impractical.

The dataset is public and available for download from its website and I was wondering if I could download it directly to Floydhub. I wrote a script that does just that and tried running it on Floydhub but after it had completed, I found no trace of the data anywhere on my account.

Going through the forums, I came across a few links that explained how to do just that (http://docs.floydhub.com/home/using_datasets/#creating-a-dataset-from-downloads) but that's giving me a 404.

Wonder if there's someway to do that.