Data licence check

Data licence check

Verify that every data source in a plan permits the intended use — especially redistribution of anything derived from it — and steer the work toward shareable data, so that others can reproduce and verify the results.

Default stance: prefer a shareable-only data selection. A result others cannot re-run on the same data is much less useful. Restricted data is allowed only for explicitly research-only runs that will never be published, and even then flag that no one else can verify them and offer a shareable alternative.

When to run this

• While planning or scoping any experiment that selects data sources.
• Before committing a data selection (e.g. inside /interview-and-plan).
• Before training, generating, or publishing anything derived from data.
• Whenever the user asks to check, audit, or reason about data licences.

The core rule for derived work

A trained model and its generated outputs are derivative works of the training data. The training data's licence therefore propagates to them. So the question is never only "may I use this text?" but "may I share a model (and its outputs) derived from it?"

Aggregate facts about the data — evaluation scores, counts, metadata, identifiers — are generally not derivative works and are normally safe to publish. The copyrighted expression (the text itself) and adaptations of it (models, generated text, aligned/reformatted corpora) are the sensitive part.

How to check

1. List every source the plan touches: each dataset, corpus, translation, scraped collection, and any pretrained model used as a base.

2. Find each licence from the authoritative field, not a proxy. Record the exact licence per source; never assume.

3. Classify each for sharing a derivative:

   Licence	Share a derived model/output?
   Public Domain / CC0	Yes
   CC-BY	Yes (attribution)
   CC-BY-SA	Yes, but ShareAlike propagates — the derived work must also be CC-BY-SA
   CC-BY-NC	Only non-commercially (opt-in), never commercially
   CC-BY-ND, CC-BY-NC-ND	No — NoDerivatives forbids sharing adaptations
   Unknown / unstated / other	No — treat as not shareable

4. Propagate the strictest constraint across the whole selection: one by-sa source makes the derived model by-sa; one by-nd/nc-nd/unknown source makes the whole run non-publishable as a derivative.

5. Report a table of source → licence → shareable? → resulting model licence, and a clear go / no-go. If any source blocks sharing, propose a shareable-only subset and note what it costs (which sources drop out).

Outcome

• Publishable / verifiable run: every source permits derivative sharing. State the propagated model licence (e.g. cc-by-sa-4.0 if any ShareAlike, else cc-by-4.0, else cc0-1.0). This is the default target.
• Research-only run: includes a non-shareable source. Allowed, but say so plainly — the model, its generated output and any side-by-side sheets cannot be published, and no one else can reproduce it. Offer the shareable subset as the alternative.

Project note: the eBible corpus

For work on DavidCBaines/ebible_corpus, the real signal is licence_Licence_Type in metadata.parquet, not the Redistributable flag — that flag covers only the source text, not a derived model. Shareable types: Public Domain, by, by-sa; by-nc is shareable non-commercially only (opt-in). Never shareable as a derivative: by-nd, by-nc-nd, Unknown, and anything else. Build licence-filtered selections up front so a run is verifiable by design, rather than discovering afterwards that a restricted translation (e.g. deuelbbk, by-nc-nd) has made the whole run unpublishable.