QBoard » Artificial Intelligence & ML » AI and ML - Tensorflow » loading mnist fashion dataset with keras

loading mnist fashion dataset with keras

  • I copied and pasted tensorflow's official Basic classification: Classify images of clothing code https://www.tensorflow.org/tutorials/keras/classification


    import tensorflow as tf
        import numpy as np
        import matplotlib.pyplot as plt
        fashion_mnist = tf.keras.datasets.fashion_mnist
        (train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

     

    and ran it. Upon running it printed a load of gibberish and wouldn't stop (almost like when you accidentally put a print in a while loop):

    Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz
        
            8192/26421880 [..............................] - ETA: 6s
           98304/26421880 [..............................] - ETA: 14s
          106496/26421880 [..............................] - ETA: 27s
          417792/26421880 [..............................] - ETA: 10s
          425984/26421880 [..............................] - ETA: 13s

     

    so I terminated it. The above is just a VERY small portion of what printed. I ran it again, only to get an error straight away.

    line 7, in <module>
            (train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
          File "C:\Users\david\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\datasets\fashion_mnist.py", line 82, in load_data
            imgpath.read(), np.uint8, offset=16).reshape(len(y_train), 28, 28)
          File "C:\Users\david\AppData\Local\Programs\Python\Python38\lib\gzip.py", line 292, in read
            return self._buffer.read(size)
          File "C:\Users\david\AppData\Local\Programs\Python\Python38\lib\gzip.py", line 498, in read
            raise EOFError("Compressed file ended before the end-of-stream marker was reached")

     

    I checked a similar question, deleted the dataset causing this error and ran it again. I waited out the gibberish it was printing out and waited for it to finish running, only for it to prematurely terminate after about 30 minutes with this error:

    Traceback (most recent call last):
          File "C:\Users\david\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\utils\data_utils.py", line 275, in get_file
            urlretrieve(origin, fpath, dl_progress)
          File "C:\Users\david\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 286, in urlretrieve
            raise ContentTooShortError(
        urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 9191424 out of 26421880 bytes>
    
    During handling of the above exception, another exception occurred:
     
        Traceback (most recent call last):
          File "C:\Users\david\Documents\DAVID\Documents\Education\Computer Science\Extra stuff\Machine learning\Neural networks\ep.2.py", line 7, in <module>
            (train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
          File "C:\Users\david\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\datasets\fashion_mnist.py", line 75, in load_data
            paths.append(get_file(fname, origin=base + fname, cache_subdir=dirname))
          File "C:\Users\david\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\utils\data_utils.py", line 279, in get_file
            raise Exception(error_msg.format(origin, e.errno, e.reason))
        Exception: URL fetch failure on https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz: None -- retrieval incomplete: got only 9191424 out of 26421880 bytes

     

    I've tried deleting that dataset many times and each time I run the code the same thing happens again and again.

    I can't find any forums or stack overflows on what to do when this happens, any help would be appreciated.

    TL;DR what do I do to load a MNIST dataset when copying and pasting tensorflow's tutorial code leads to errors?

      December 11, 2021 3:31 PM IST
    0
  • load_data function

    tf.keras.datasets.fashion_mnist.load_data()
    

     

    Loads the Fashion-MNIST dataset.

    This is a dataset of 60,000 28x28 grayscale images of 10 fashion categories, along with a test set of 10,000 images. This dataset can be used as a drop-in replacement for MNIST.

    The classes are:

    Label Description
    0 T-shirt/top
    1 Trouser
    2 Pullover
    3 Dress
    4 Coat
    5 Sandal
    6 Shirt
    7 Sneaker
    8 Bag
    9 Ankle boot

    Returns

    • Tuple of NumPy arrays(x_train, y_train), (x_test, y_test).

    x_train: uint8 NumPy array of grayscale image data with shapes (60000, 28, 28), containing the training data.

    y_train: uint8 NumPy array of labels (integers in range 0-9) with shape (60000,) for the training data.

    x_test: uint8 NumPy array of grayscale image data with shapes (10000, 28, 28), containing the test data.

    y_test: uint8 NumPy array of labels (integers in range 0-9) with shape (10000,) for the test data.

    Example

    (x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
    assert x_train.shape == (60000, 28, 28)
    assert x_test.shape == (10000, 28, 28)
    assert y_train.shape == (60000,)
    assert y_test.shape == (10000,)
      December 14, 2021 11:57 AM IST
    0
  • Fashion-MNIST can be used as drop-in replacement for the original MNIST dataset (10 categories of handwritten digits). It shares the same image size (28x28) and structure of training (60,000) and testing (10,000) splits. It’s great for writing “hello world” tutorials for deep learning.

    import tensorflow as tf
    import numpy as np
    import matplotlib.pyplot as plt
    # Load the fashion-mnist pre-shuffled train data and test data
    (x_train, y_train), (x_test, y_test) = tf.keras.datasets.fashion_mnist.load_data()
    print("x_train shape:", x_train.shape, "y_train shape:", y_train.shape)​
      December 16, 2021 12:53 PM IST
    0
  • # example of loading the fashion mnist dataset
    from matplotlib import pyplot
    from keras.datasets import fashion_mnist
    # load dataset
    (trainX, trainy), (testX, testy) = fashion_mnist.load_data()
    # summarize loaded dataset
    print('Train: X=%s, y=%s' % (trainX.shape, trainy.shape))
    print('Test: X=%s, y=%s' % (testX.shape, testy.shape))
    # plot first few images
    for i in range(9):
    	# define subplot
    	pyplot.subplot(330 + 1 + i)
    	# plot raw pixel data
    	pyplot.imshow(trainX, cmap=pyplot.get_cmap('gray'))
    # show the figure
    pyplot.show()
      December 20, 2021 12:02 PM IST
    0