This means that arrays in the JSON are not added as tables if they are empty. Therefore, using this data provider on an existing template will break as soon as the JSON data contains empty arrays. (The error is “The project uses the following tables that do not exist: …”)
Am I missing something here? What’s the reason empty tables are removed?
The provider would need array elements in order to pass them as fields for the table. Thus, the structure of the JSON, at least for the first record (which gets parsed as structure reference) needs to stay the same. Maybe the addition of a “schema” JSON file would be a good addition for cases where empty arrays may occur? Feel free to add this as suggestion to the idea place category.
Thanks for the reply!
Yes, so if I understand correctly, the fields / objects can (of course) not be inferred correctly if the arrays are empty. So the data source and template won’t match…
Do you have a suggestion how I can work around this?
I mean I do have the schema, but I don’t know how to tell LL about these fields and empty tables. (apart from writing a new IDataProvider from scratch myself).
As a last resort I’m thinking about putting the data into a in-memory SQLite DB. To my understanding that should work for sure. But I’m hoping to avoid this, as I expect it will be quite slow.
Indeed we’d need to provide a way to pass a schema JSON in order to fix this. Stay tuned, this totally makes sense and shouldn’t be too hard to implement. May we contact you in order to test a possible implementation against your use case?
I reimplemented the IDataProvider for JSON with schema support myself now (using NJsonSchema), it seems to work now. Not yet sure about all corner cases though.
I can send you the implementation for reference later if you’re interested (once I’m done cleaning up)…
Unfortunately, @Fabian_Schneiter did never reply here. We have meanwhile finished our own project using NJsonSchema and plan to release it on GitHub in January 2022. If you’d like to test with the early bits feel free to DM me.