Bdtjtk

Im at least assuming something is screwing that up...
here is what is going on. I am making ML models for currency pairs in the foreign exchange market. I make one model for each pair im looking at. So, I have 8 models.

I do the following:

1.) make a model based on data, from a csv file, for each pair. kind of like this:

namelist = ['usdcad', 'eurjpy', 'usdjpy', 'gbpjpy', 'audusd', 'usdchf', 'nzdusd', 'gbpusd', 'eurusd']





for filename in namelist:



with open(filename+"/all.csv",'r') as dest_f:

    data_iter = csv.reader(dest_f,

                           delimiter = ",",

                           quotechar = '"')

    data = [data for data in data_iter]



    ##do some stuff to organize the data



    model50 = linear_model.LinearRegression()

    model50.fit(X, Y50)



    afile = open(r''+filename+'.pkl', 'wb')

    pickle.dump(model50, afile)

    afile.close()

Then in another file I load realtime data, which I use as input for my model, I save that data to csvs

namelist = ['USDCAD', 'EURJPY', 'USDJPY', 'GBPJPY', 'AUDUSD', 'USDCHF', 'USDCAD', 'NZDUSD', 'GBPUSD', 'EURUSD']



for name in namelist:

CSV_URL = 'REMOVED BECAUSE IT HAS MY KEY'+name



with requests.Session() as s:

    download = s.get(CSV_URL)



    decoded_content = download.content.decode('utf-8')



    cr = csv.reader(decoded_content.splitlines(), delimiter=',')

    my_list = list(cr)

    del my_list[:18]

    del my_list[-1]

    with open(name+".csv", "wb") as f:

        writer = csv.writer(f)

        writer.writerows(my_list)       

Then finally, I run all my predictions at once:

namelist = ['USDCAD', 'EURJPY', 'USDJPY', 'GBPJPY', 'AUDUSD', 'USDCHF', 'USDCAD', 'NZDUSD', 'GBPUSD', 'EURUSD']





for name in namelist:



with open(name+".csv",'r') as dest_f:

    data_iter = csv.reader(dest_f,

                           delimiter = ",",

                           quotechar = '"')

    res = [data for data in data_iter]





res = [[x[3],x[4],x[5],x[6]] for x in res]

input = np.zeros((1,len(res)*4))

indx = 0

while 1:

    for row in res:

        for col in row:

            input[0,indx] = float(col)

            indx += 1

    break   



pname = name.lower()    



file = open(r''+pname+'.pkl', 'rb')

mymodel = pickle.load(file)

file.close()    



prediction = mymodel.predict(input) 

sorry about any small formatting errors, I am still pretty bad at using this site properly.

anyways, so what actually happens here is that two of my predictions are correct, clearly the model is working right. the other 6 predictions make NO SENSE AT ALL. It's like it's trying to use one of the other models, but it shouldn't be.... Or like the wrong model gets pickled to a filename. I've examined the inputs, the files, they are all correct. Furthermore, using the exact same CSV files, I used the exact same techniques to make a model and predict JUST usdcad, and it works great! But the usdcad prediction comes out totally wrong when I use the scripts below.

Does anyone know what is going wrong here? I can't figure it out. Thanks.

**QUICK EDIT:
I've confirmed that the problem is that it is with the pickled models. It's overwriting them or keeping old model objects in memory or something...

EDIT2: Some extra info
I have discovered that if I pickle my model, and then load it, and then use it - all in the same script, then it works fine. If I pickle my model, and then load it in another script (using the exact same code), it does not work correctly. This is the case even when I only make one model.