feat: Milestone 5 - workflow manifest integration and example workflows

[DhanashreePetare]

Pull Request

Description

Integrates ManifestContext into the workflow engine, so workflow run --manifest path.jsonld (or a manifest: key inside the YAML file itself) produces one unified manifest covering every step in the run, not just a single command. Each manifest file entry is tagged with dbus:stepName so multi-step runs stay traceable to which step produced or failed on which file. A readable console summary is now printed automatically after every workflow run, no flag required.

Adds five tested, real-world example workflows matching the proposal's named use cases: reproducible research download, automated nightly publishing pipeline, batch deployment with retry, CI/CD integration, and failure debugging.

Related Issues
Issue #80

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• This change requires a documentation update
• Housekeeping

Checklist:

• My code follows the ruff code style of this project.
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (if applicable)
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
  • poetry run pytest - all tests passed
  • poetry run ruff check - no linting errors

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5a8b9ae2-4ffb-49f4-9732-ca64feec9b52

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Integer-Ctrl]

Overall error on replaying manifest produced by example workflows
databusclient manifest replay ./manifests/[...].jsonld Error: Manifest field dbus:replayParams must be an object.

[Integer-Ctrl]

Not working

(databus-python-client) fhofer@pop-os ~/ddrive/dev/infai/dbpedia/databus-python-client (remotes/dhanashree/gsoc-2026*?) $ databusclient workflow run ./examples/workflows/reproducible-research-download.yml
SPARQL endpoint https://databus.dbpedia.org/sparql
Downloading version: https://databus.dbpedia.org/DhanashreeP/test-group/workflow-source-data/1.0
Download file: https://databus.dbpedia.org/DhanashreeP/test-group/workflow-source-data/1.0/workflow-source-data_type=swagger.yml
Redirects url:  https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47.4k/47.4k [00:00<00:00, 6.00MiB/s]
Workflow complete.
  fetch_research_data: success

Command  : workflow
Executed : 2026-08-19T15:58:12.924409+00:00
Files    : 1 succeeded · 0 failed
Status   : completed

Manifest written to ./manifests/research-download.jsonld

(databus-python-client) fhofer@pop-os ~/ddrive/dev/infai/dbpedia/databus-python-client (remotes/dhanashree/gsoc-2026*?) $ databusclient manifest replay ./manifests/research-download.jsonld    
Error: Manifest field dbus:replayParams must be an object.

dbus:replayParams missing in manifest file

[Integer-Ctrl]

The proposal states

A deploy operation partially fails across
multiple files. Because the manifest records structured error details — including the error
message, traceback, and retry count — directly against each failed file entry, running
databusclient manifest summary run.json gives an immediate, structured
view of exactly what failed and why. Debugging drops from hours of log reading to a
single command.

But the summary does not reveal insights

(databus-python-client) fhofer@pop-os ~/ddrive/dev/infai/dbpedia/databus-python-client (remotes/dhanashree/gsoc-2026*?) $ databusclient workflow run ./examples/workflows/failure-debugging.yml          
SPARQL endpoint https://databus.dbpedia.org/sparql
Downloading version: https://databus.dbpedia.org/DhanashreeP/test-group/workflow-source-data/1.0
Download file: https://databus.dbpedia.org/DhanashreeP/test-group/workflow-source-data/1.0/workflow-source-data_type=swagger.yml
Redirects url:  https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47.4k/47.4k [00:00<00:00, 5.63MiB/s]
Workflow failed.
  fetch_source: success
  deploy_with_bad_key: failed

Command  : workflow
Executed : 2026-08-19T16:07:50.138975+00:00
Files    : 1 succeeded · 1 failed
Status   : completed with errors

Manifest written to ./manifests/failure-debug.jsonld
Error: Step 'deploy_with_bad_key' failed: Could not deploy dataset to databus. Reason: 'Authentication failed.'
(databus-python-client) fhofer@pop-os ~/ddrive/dev/infai/dbpedia/databus-python-client (remotes/dhanashree/gsoc-2026*?) $ databusclient manifest summary ./manifests/failure-debug.jsonld        
Command  : workflow
Executed : 2026-08-19T16:07:50.138975+00:00
Files    : 1 succeeded · 1 failed
Status   : completed with errors