Sources indexing in MongoDB #73
Description
Context
In order to monitor specific data sources we need to implement a way to monitor the status of these sources. The user can define a set of data sources (data-endpoints?) that are going to be checked automatically for updates.
This can be used to trigger a new execution when the source is updated.
Proposed solution
Separate indexing from monitoring. Sources are a new mongoDB collection that keeps the link to the data source, the type, the last time it was accessed and other metadata of relevance. This collection can be at the same level of components.
Acceptance criteria
- New MongoDB collection called
sources -
sourcesschema. - Add a new source in CLI
- Add a new source in GUI
- Listing sources in CLI
- Listing sources in GUI