Hi Jordan. Thanks for your interest in nuScenes. We apologize for these problems.

1. Regarding the XML error, this seems to be a rare communication issue between our hoster and AWS. See this for more details. We are working on implementing that fix. Meanwhile it's good you tried another machine.

2. I can confirm your issue with WinRAR not recognizing the file format properly. A temporary workaround is to rename the .tar files to .tbz2. That works for me. We will rename these files on our end soon.

Let me know if you have any other questions.

I tried your fix for windows by renaming the file extension. It worked for me! Thank you. I am excited to work with this data set.

I am now getting the error on a windows machine using winrar data\nuscenes\nuscenes_teaser_pointclouds_samples_v1.tbz2: The archive is corrupt and data\nuscenes\nuscenes_teaser_pointclouds_sweeps_v1.tbz2: The archive is corrupt
. This is happening to all of the folders except the metadata file. Most of them error out before half of the data is extracted. When I try to download the data again to see if it was corrupted during download I get the same This XML file does not appear to have any style information associated with it error again. I know you are working on this issue, but I think the first issue may be related in some way.

I am now getting the error on a windows machine using winrar data\nuscenes\nuscenes_teaser_pointclouds_samples_v1.tbz2: The archive is corrupt and data\nuscenes\nuscenes_teaser_pointclouds_sweeps_v1.tbz2: The archive is corrupt
. This is happening to all of the folders except the metadata file. Most of them error out before half of the data is extracted. When I try to download the data again to see if it was corrupted during download I get the same This XML file does not appear to have any style information associated with it error again. I know you are working on this issue, but I think the first issue may be related in some way.

Hello jtunnell, you need to refresh the page to download these files again. The links will usually expire in 10 minutes or so, refresh the page to get new links, thanks!

I have downloaded the data again today where I made sure that I hard reloaded the links. I don't experience the XML file error but I am still getting the error data\nuscenes\nuscenes_teaser_pointclouds_samples_v1.tbz2: The archive is corrupt. This is happening for all of the folders except for the meta. If you say that the links expire in 10 minutes and it takes much longer to download then 10 minutes would that be causing the issue here? I don't think it would be, but it might be. It seems that the download isn't working completely as I am seemingly only able to extract only a portion of the data successfully. Thank you for your help.

I mean once you start downloading, you should be able to download the whole file. It can be that the download was not completed, usually it is caused by network issue. Do you use browser to download or something else? @jtunnell

Maybe you can try wget, for example:

# wget -c -O filename "url", for example:
wget -c -O nuscenes_teaser_images_samples_v1.tbz2 "copy_and_paste_your_url_here"

-c means continue, so if your connection is broken, you can refresh the page and get another url and continue your downloading.

I will give wget a try. In chrome, I get the indication that I get the entire download for every file. However, it always breaks when I extract it. I will try this on a few machines tomorrow.

I used the wget method to download the lidar and radar point clouds samples and I still got the same issue. Is it possible that something is wrong with the file currently? Also, a question about the radar what raw radar information do we get. Can we read the files our selves or do we have to use your code to work with the data?

I mean you need to make sure that you download the whole file.

For example, from wget output, you can see how big the file is and what the progress is, I suggest you check file size on your local disk. The whole point of using wget is you can continue downloading the file from where it was left.(But be careful with the command, use the corresponding url for the file)

@holger-nutonomy I suggest we show file length and md5sum or sha1sum on download page.

@Qiang-Xu Yes, let's show md5sums.

@jtunnell
The radar data comes in the standardized Point Cloud Data format file. But the best is to use our code to read it.

I have used wget on a few computers and I get the full download. I have double checked that I have used the correct commands. I don't think this is an issue of getting the whole file. I have also tried downloading from your webpage again. Every time I extract the data I get the issue data\nuscenes\nuscenes_teaser_pointclouds_samples_v1.tbz2: The archive is corrupt. It seems to always happen here for this file

Hey @jtunnell, sorry for the delay, I'm on a business trip this week. I wonder if you can create an account for me on any of your linux machines that I can ssh in or you can spin up a VM somewhere. I'd like to test it in your environment, because on my end it is okay. Let me know your thoughts.

@jtunnell I'm closing this issue for now, as we cannot reproduce it. Please let us know if you continue to have problems.

I have the same "corrupted archive" issue in Windows.

I try: sudo wget v1.0-trainval02_blobs.tgz https://s3.amazonaws.com/data.nuscenes.org/public/v1.0/v1.0-trainval02_blobs.tgz?AWSAccessKeyId=AKIA6RIK4RRMFUKM7AM2&Signature=XXXXXXXXXXXXXXXXXXXXXXX&Expires=1560083334 but it does work. What I am doing wrong?

@ChFernandez12 Can you try again with a new key? The key expires after a while if you don't start (!) the download.

Yes I have try it several times with different keys from my ssh server. Any thoughts?

@ChFernandez12 I've just tried this and it seems that the double quotes around the URL are needed. So please do
wget -c -O v1.0-trainval02_blobs.tgz "https://s3.amazonaws.com/data.nuscenes.org/public/v1.0/v1.0-trainval02_blobs.tgz?AWSAccessKeyId=AKIA6RIK4RRMFUKM7AM2&Signature=XXXXXXXXXXXXXXXXXXXXXXX&Expires=1560083334"

It worked! Thank you!

After making an account to download the data, I found that the button is grey and I cannot download the data. How should I do to get the nuScene and nuImage data sucessfully? Please help me!

Thanks.
If I want to conduct 3D LiDAR-image object detection experiment, which data need I download? Or say, I am confused about the difference between nuScenes-lidarseg and Full dataset(v1.0). I do not know which one is the LiDAR data corresponding image in nuImage subset.

nuImage is a separate dataset that is 2d only.
nuScenes is the dataset with 3d object annotations.
nuScenes-lidarseg is an extension for nuScenes with lidar point-level labels.

Thanks. So, you mean if I want to test my 3d detector performance, 'full dataset v1.0' is suitable for me?

I do not find 'nuScenes' name in the official downloading website.

Exactly. Mini is good if you just want to run it on a small subset (10 scenes).

Thanks for your kindness, and there are still two problems confused me as follows:

1. If I want to submit the test result to the evaluation server, should I download the whole trainval and test data under 'the full dataset v1.0'?
2. If I have download the whole trainval and test data and want to construct the following file structure, how should I do?
   Each samples and sweeps directories should be extracted from the subset, e.g. v1.0-trainval01_blobs, v1.0_trainval02_blobs, and then move to a unified 'samples' or 'sweeps' directory. Does it?
   

Thanks again for any instructions.

1. If you already have a model and you just want to run it, you can use test only.
2. Yes, depending on which "zip" program you use, you need to merge all of the folders of the different archives.

1. If you already have a model and you just want to run it, you can use test only.
2. Yes, depending on which "zip" program you use, you need to merge all of the folders of the different archives.

Thanks.

1. I mean if I want to train a model and submit its test result to the evaluation server, need I download the whole trainval data to train my model? Or say, if I only use mimi data for training, does the detection performance damage? Maybe I think it does, but do not determine.

Yes, 10 scenes are not a lot (200s of data). Hence the mini split performance will be pretty bad.
You will need to train on the entire train (or trainval) set.

Hello,

When I click on the US link to download the dataset I get the following console message when I inspect in Chrome instead. Any idea why I can't download? thanks

Access to XMLHttpRequest at 'https://o9k5xn5546.execute-api.us-east-1.amazonaws.com/v1/archives/nuplan-v1.0/nuplan-maps-v1.0.zip?region=us&project=nuScenes' from origin 'https://www.nuscenes.org' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
instrument.ts:129 err Error: Network Error
at e.exports (createError.js:16:15)
at h.onerror (xhr.js:99:14)
at XMLHttpRequest.r (helpers.ts:88:17)
trycatch.ts:281 GET https://o9k5xn5546.execute-api.us-east-1.amazonaws.com/v1/archives/nuplan-v1.0/nuplan-maps-v1.0.zip?region=us&project=nuScenes net::ERR_FAILED 404

@holestine if this is an issue related to the nuPlan dataset, pls post it at https://github.com/motional/nuplan-devkit