Anyone got solutions or hints for automatic persistent serialisation?
Usecase
I want to save and load data made over many release versions of an application, potentially with updates years apart.
The data itself is records, spreads of records, dictionaries of records.
Whilst I try and keep changes minimal there could occasionally be major structural changes to the data schema.
Basically I want to ensure as much automatic backwards compatibility as possible when loading data created with an old version of the app.
And then minimise my manual programming effort to only handle the difficult cases.
I already encode an app version number in the data so I will know if it’s attempting to load data from an old version.
Problem
The automatic serialisation of vvvv gamma is great and usually a good workflow. But as you read on the Gray book serialisation page the underlying fsPickler library is not suitable for persistent use, a future version may interpret data differently.
I tried manual serialisation but I found on a large structure it’s very laborious and prone to programmer error once you have a few versions going.
So I want to use an automatic workflow as far as possible.
Maybe I even need a schema checker that tells me what changes I made between versions?
While I can easily conceive the changes I made when I update the app from version A to version B, once I am working on app version H and I want to load data created with app version A I’m much less sure.
Questions
Breaking it down I think I can always use built in serialisation for saving. FSpickler will ensure the properties are saved in a valid common format (JSON, XML), particularly if I use Include Defaults.
(In principle this means I could switch to use something other than fsPickler for deserialising in future.)
And then if I load data created with the current version of the app it can also use the built in deserialisation.
I only need to have special handling for the case of loading data from previous versions.
Does that sound right?
Then I would love to hear thoughts or solutions from others towards this problem.
Anyone use any schema checking?
Is there any solution that would tell you meaningful differences between data and schema?