---
name: fabric-ops-f341359dcc9c
description: Operate and maintain a Fabric data platform — orchestrate pipeline DAGs, run VACUUM, update platform inventory, manage workspace items, and execute operational routines. Use for day-to-day platform operations, maintenance windows, GDPR purges, or environment setup.
---

# fabric-ops

## MUST

- Run VACUUM weekly on all Delta tables (retention_hours=168)
- Document every scheduled pipeline as a graph memory node (`graph_create_node`)
- Update platform inventory after creating or deleting Fabric items
- Never run `VACUUM RETAIN 0 HOURS` unless explicitly purging toxic or GDPR-flagged data

## PREFER

- `fabric-vibe` subcommands (which wrap the Fabric CLI) over direct `fab` calls for item management
- Idempotent setup scripts (running twice causes no harm)

## AVOID

- Manual portal-only operations without a corresponding CLI command (creates undocumented state)
- Running Gold jobs before Silver is confirmed successful (violates DAG ordering)
- VACUUM with retention <168 hours in normal operations

## Maintenance Routine

**Daily**: Check pipeline run status, triage failures, review DQ notebook results  
**Weekly**: Run VACUUM, check for schema drift, review slow jobs, review DQ trend reports  
**Monthly**: Capacity review, access review, Key Vault secret rotation check, stale item audit

## Lakehouse Inspection

```bash
# List all lakehouses in the workspace with their tables and column schemas
fabric-vibe lakehouse list-tables

# Scope to a specific lakehouse
fabric-vibe lakehouse list-tables --lakehouse bronze_lh

# Inspect one table
fabric-vibe lakehouse list-tables --lakehouse bronze_lh --table raw_orders

# Machine-readable JSON (pipe to jq for filtering)
fabric-vibe lakehouse list-tables --json
```

Column schema is read from each table's Delta transaction log via the OneLake DFS
endpoint using the Fabric CLI credential. If the token is unavailable, table
names and types are still listed without schema.

## Daily Checks

```bash
# List items in the workspace (shows notebooks, lakehouses, etc.)
fab api "workspaces/$FABRIC_WORKSPACE_ID/items" --output_format json

# Check recent job runs for a specific notebook item
fab api "workspaces/$FABRIC_WORKSPACE_ID/items/<item_id>/jobs/instances" --output_format json

# Monitor a specific job instance
fabric-vibe notebook deploy monitor "$FABRIC_WORKSPACE_ID" <item_id> <job_instance_id>
```

Check DQ notebook run results in the Fabric portal (Activities → Notebook runs).

## VACUUM Pattern

```python
from delta.tables import DeltaTable

tables_to_vacuum = [
    f"abfss://bronze@onelake.dfs.fabric.microsoft.com/{lakehouse_id}/Tables/raw_orders",
    f"abfss://silver@onelake.dfs.fabric.microsoft.com/{lakehouse_id}/Tables/silver_orders",
]

for path in tables_to_vacuum:
    dt = DeltaTable.forPath(spark, path)
    dt.vacuum(retention_hours=168)
    print(f"✓ VACUUM complete: {path}")
```

## DAG Orchestration (Data Factory)

1. Bronze notebook → Silver notebook (trigger on success only)
2. Silver notebook → Gold notebook (trigger on success only)
3. On failure: send alert, do NOT cascade to next layer

## Environment Setup

```bash
# Create folder structure for local development
mkdir -p workspace data/sandbox logs

# Copy environment template
cp .env.example .env

# Discover and select workspace/resource IDs from the registry
fabric-vibe workspace init
fabric-vibe workspace switch list
fabric-vibe workspace switch <displayName>

# Install Fabric CLI
uv tool install ms-fabric-cli

# Authenticate (normally done once by tool/setup/setup.{sh,ps1})
fab auth login
```

## Platform Inventory Update

After creating a new Fabric item, add it to `memory/platform.md`:
```markdown
## Lakehouses
| Name | Workspace | Layer | Created | Notes |
|---|---|---|---|---|
| bronze_lh | fabric-sandbox | Bronze | 2026-05-06 | Raw + masked ingest tables |

## Source Systems
| Name | Type | Env Var Prefix | Cadence | Sensitive Fields |
|---|---|---|---|---|
| ORDERS | file | SRC_ORDERS | ad hoc sandbox | customer_name, email |
```


---
Source attribution: scardoso-lu
Original project: https://github.com/scardoso-lu/fabric-skills-settings/blob/33534eb185fda573125308a0129457bd65419d13/server/skills/fabric-ops/SKILL.md
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Mega AI Hub.

## Upstream license notice

MIT License

Copyright (c) 2026 Silvio Cardoso

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



# Mega AI Hub resource metadata

Creator / publisher: scardoso-lu
Original project: https://github.com/scardoso-lu/fabric-skills-settings/blob/33534eb185fda573125308a0129457bd65419d13/server/skills/fabric-ops/SKILL.md
Attribution basis: Source repository record

Fields of work: Development & Engineering
Roles: Backend Developer, Data Engineer, DevOps / SRE
Specialties: Backend, DevOps
Tools: Fabric
Country applicability: Country not specified
Classification basis: Source catalog labels
Instruction SHA-256: da90fc638719e3aea9120636128b36c7256d74b78df02e08e5f453794476b30d