pickles, objects and .tox updates

I’m working on a Python heavy application for inventory (I know, not the most exciting thing Touch can do), which stores class objects using components’ dictionaries; basically everything in inventory is an object, and objects move to different dictionaries depending on what venues they’re in, etc. That’s all good, but if I edit an execute DAT that creates these objects, then save Touch, I get this:

Warning: Error saving storage for operator /project1/container3/workingStock:_pickle.PicklingError: Can’t pickle <class ‘/project1/container3/wStockMgmg/addDevices/chopexec4.DeviceEach’>: it’s not the same object as /project1/container3/wStockMgmg/addDevices/chopexec4.DeviceEach

So I basically understand what this is, that the objects already created can’t be pickled. And right now, that’s not a super big deal, because I’m still developing this, so when I open touch back up, I don’t really care that I lost my inventory. But this could be a really big problem when it comes to updates. So my question is, how do you achieve data persistence? how do you preserve class objects and other parameters through updates?

Also, as a side, and this is beyond the realm of TD… where do objects go? If I’m not mistaken, pickling them stores them on the computer’s drive in byte form. But what if I pickle a bunch of stuff in the command line and then never retrieve it? What if I pickle things in a TD session, and then delete the .toe? are there just objects lost to the computational abyss?

To answer the second question, the pickled data is stored in the .toe file when pickling inside of TD.

That makes sense, and if I open a verison of the .toe saved before the execute DAT is altered, those objects are still there. In that case, is there a way to extract/import those objects into a new .toe or .tox?

TouchDesigner is not designed to store custom Python objects. Your best bet is to either store your data using builtin Python objects (like dicts) which can be saved in storage automatically, TouchDesigner objects (like tableDATs) which save in .toe automatically, or develop your own saving method for objects (JSON is often great for this) which needs to be written to disk manually. Then you can load that data into you custom classes when you start your .toe file.

I’ve been playing around with pickling the objects myself, so the script that creates the objects also automatically pickles them, or the dictionaries they’re stored in. I’ve also moved all my classes to a single DAT that sits at the top of my network, and import from that. I’d like to import from an external file, but I’m having trouble telling Touch exactly where to look for the module.

I’ll check out JSON once I start feeling more comfortable with Python, though.