Validate BIDSLayout

[adelavega]

Currently, BIDSLayout is not validated, because model files are not valid BIDS (see: #204).

This can cause other unintended problems, so it would be best to add a custom config to pass initi pybids for these files that don't pass validation.

[effigies]

Doing this one now.

[effigies]

What all sorts of files do we need to whitelist? Is it just model files?

[effigies]

Also, if we want validate=True to work, then we're going to need to update bids_validator.

[adelavega]

Were you previously using validate=False when loading a layout prior to pybids.Analysis? There's a chance that my bundle event directories are not fully BIDS compliant and won't pass validation.

I'll check

[effigies]

I think validation may get passed over when explicit includes are used.

[adelavega]

I'm not using explicit includes I don't think, just passing a bundle directory with extra event files as a derivative, and I'm not 100% its BIDS-valid, although I think it probably is, since validation used to be True for that step in fitlins, no?

[adelavega]

In a quick test, it seems to work (I can build an Analysis object and set it up) with a validated BIDSLayout.

AFAIK I think it was just the model files, then

[effigies]

Okay. So that's going to need an updated validator.

One possible hack is to allow BIDSLayout to take a BIDSValidator()-like object instead of instantiating its own on True, and then monkey-patch a validator in FitLins before passing it in. That may wreak minor havoc on saved initialization arguments for the database file, though...

[adelavega]

Your previous suggestion wont work?

What if we enable validation but package our own config that handles smdl suffixes and model entities?

[effigies]

No, because BIDSValidator configs are hard-coded. See https://github.com/bids-standard/bids-validator/issues/911.

[adelavega]

Ah. Well an alternative is to create a new BIDSLayout only in the instance you were previously having to turn validation off. How costly would that be?

Looks like previously you were creating a BIDSLayout and only using it for a single query, which really only applied to a very specific use case (not passing in a model file explicitly):

if not isdefined(models):  
    models = layout.get(suffix='smdl', return_type='file')

So you could maybe avoid using a BIDSLayout here, or use exclude/include to make this indexing as narrow as possible (and only create a layout if models are not defined). This would minimize the impact of creating a new layout here.

Or if you insist on the monkey-patching, you could do it only here, as to not mess with the main layout/saved initialization arguments. But I would suggest avoiding this approach because its yet another change to the BIDSLayout API for a niche use case.

For all other uses we would load from validate BIDSLayout db.