Bdtjtk

I am trying to convert a .npz file to .csv format, but it is giving the following key error
KeyError: '0 is not a file in the archive'

I had a sparse matrix which I converted to .npz format. I then loaded the npz file using np.load(). I tried converting the loaded npz file to csv using np.savetxt() but it gives the following error
KeyError: '0 is not a file in the archive'.

What does this key error mean and how to solve it?

I tried the following code:

DF = np.load("DF_tfidf.npz")



np.savetxt("DF.csv",DF)

You cannot convert NPZ file to csv file. First we need to find out what are the files in NPZ File like below

np_Array=np.load('DF_tfidf.npz')

print(np_Array.files)

for example if output is like ['arr_0'] for above print
So you need to extract that array and then convert it to csv like below.

arr=np_Array.files[0]

np.savetxt("DF.csv", np_Array[arr], delimiter=",")

This isn't a problem of how to convert npz to csv, but how to properly load the data from the npz, and then save that as csv. In general a npz is a file archive that contains several arrays. A csv on the other is a format for saving one 2d array.

You could, in theory, write each file of the npz to its own csv. But if the npz saves some complex object, rather than a random set of array, that's probably not what you want to do. My guess is that you have a scipy.sparse matrix (possibly created in the course of some machine learning project). In that case you should focus on how to write a sparse matrix, or some representation of it, not on converting its npz save.

Let's make a scipy sparse matrix and save it:

In [45]: from scipy import sparse

In [46]: M = sparse.random(4,4,.2,'csr')

In [47]: M

Out[47]: 

<4x4 sparse matrix of type '<class 'numpy.float64'>'

    with 3 stored elements in Compressed Sparse Row format>

In [48]: M.A

Out[48]: 

array([[0.30442216, 0.        , 0.        , 0.        ],

       [0.29783572, 0.        , 0.        , 0.        ],

       [0.        , 0.        , 0.83881939, 0.        ],

       [0.        , 0.        , 0.        , 0.        ]])

In [49]: sparse.save_npz('sparse.npz',M)

Now load it:

In [50]: sparse.load_npz('sparse.npz')

Out[50]: 

<4x4 sparse matrix of type '<class 'numpy.float64'>'

    with 3 stored elements in Compressed Sparse Row format>

That's the same thing that we saved.

Now look at it with np.load:

In [51]: data = np.load('sparse.npz')

In [52]: list(data.keys())

Out[52]: ['indices', 'indptr', 'format', 'shape', 'data']

In [53]: data['indices']

Out[53]: array([0, 0, 2], dtype=int32)

In [54]: data['indptr']

Out[54]: array([0, 1, 2, 3, 3], dtype=int32)

In [55]: data['format']

Out[55]: array(b'csr', dtype='|S3')

In [56]: data['shape']

Out[56]: array([4, 4])

In [57]: data['data']

Out[57]: array([0.30442216, 0.29783572, 0.83881939])

I can save the dense equivalent of this sparse matrix to a csv with:

In [60]: np.savetxt('sparse.csv', M.A, fmt='%10f',delimiter=',')

In [61]: cat sparse.csv

  0.304422,  0.000000,  0.000000,  0.000000

  0.297836,  0.000000,  0.000000,  0.000000

  0.000000,  0.000000,  0.838819,  0.000000

  0.000000,  0.000000,  0.000000,  0.000000

For a small matrix like this that's no problem. But often in machine learning the sparse matrix is very large, and M.A raises a MemoryError.

I suppose one could try to write a 3 column csv with the row,col,data attributes of a coo format matrix, the same sort of numbers we get with:

In [62]: print(M)

  (0, 0)    0.3044221604204369

  (1, 0)    0.29783571660339536

  (2, 2)    0.8388193913095385