feat: add OpenCode analysis workflow with Devin integration #2152
Conversation
- Add analyze.sh script for code analysis using OpenCode CLI with Plan agent - Add create-devin-sessions.sh for parsing findings and creating Devin sessions - Add GitHub Actions workflow with cron schedule (weekly) and manual trigger - Support max 3 concurrent analyses - Add README documentation for the workflow Co-Authored-By: yujonglee <yujonglee.dev@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
✅ Deploy Preview for hyprnote ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for hyprnote-storybook ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
📝 WalkthroughWalkthroughExtended GitHub Actions workflow with matrix-based analysis targets and manual triggers. The Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…un setup Co-Authored-By: yujonglee <yujonglee.dev@gmail.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (5)
scripts/opencode-analysis/analyze.sh (1)
72-74: IFS-based parsing may fail if colon appears in description.Line 73 uses
IFS=':'to parse target entries. If a description contains a colon (e.g., "Plugin: for X"), the split will break. Consider using a more robust delimiter or format (e.g.,@or structured data).-TARGETS=( - "plugins/listener:Audio capture and session management plugin" +TARGETS=( + "plugins/listener@Audio capture and session management plugin"Then update line 73:
- IFS=':' read -r target description <<< "$target_entry" + IFS='@' read -r target description <<< "$target_entry".github/workflows/opencode_analysis.yaml (1)
24-29: Quote $GITHUB_PATH for defensive coding.Line 26 appends to
$GITHUB_PATHwithout quoting. While$GITHUB_PATHis unlikely to contain spaces in the GitHub Actions environment, defensive quoting is a best practice:- echo "$HOME/.opencode/bin" >> $GITHUB_PATH + echo "$HOME/.opencode/bin" >> "$GITHUB_PATH"This aligns with the shellcheck SC2086 warning and prevents edge-case failures if environment variables are unexpectedly modified.
scripts/opencode-analysis/create-devin-sessions.sh (3)
10-32: Add error handling for malformed JSON in parse_findings.Lines 15–27 iterate over JSON files and attempt to parse them with
jqwithout verifying that the input is valid JSON. Ifanalyze.shproduces corrupted output (as flagged in the earlier critical issue), line 20'sjq -e '.error'will fail silently, and line 25'sjqoperation will error, halting the script or producing an incompletefindings.json.Add a pre-flight validation:
for file in "$OUTPUT_DIR"/*.json; do if [[ -f "$file" ]]; then local target_name=$(basename "$file" .json) local content=$(cat "$file") + # Validate JSON structure before processing + if ! echo "$content" | jq -e . > /dev/null 2>&1; then + echo "Skipping $target_name: invalid JSON output" + continue + fi + if echo "$content" | jq -e '.error' > /dev/null 2>&1; then echo "Skipping $target_name: analysis failed" continue fi ...
34-52: Add error handling for Devin API response parsing.Line 43 checks for
.session_idin the response without first validating that the response is valid JSON. If the Devin API returns an error or non-JSON response,jqwill fail and the script will error. Add a pre-flight JSON validation:create_devin_session() { local prompt="$1" local response response=$(curl -s -X POST "$DEVIN_API_URL" \ -H "Authorization: Bearer $DEVIN_API_KEY" \ -H "Content-Type: application/json" \ -d "{\"prompt\": $(echo "$prompt" | jq -Rs .)}") + # Validate JSON response before parsing + if ! echo "$response" | jq -e . > /dev/null 2>&1; then + echo "Failed to create session: invalid JSON response" + return 1 + fi + if echo "$response" | jq -e '.session_id' > /dev/null 2>&1; then ...
70-98: Consider splitting declare and assign to improve error capture (optional).Shellcheck flags multiple SC2155 warnings (lines 17, 18, 44, 45, 65, 66, 71, 72) recommending that variable declarations and assignments be split. While this is valid style guidance, in "Chill" review mode these are low-priority. If you wish to address them, split assignments like:
-local target_name=$(basename "$file" .json) +local target_name +target_name=$(basename "$file" .json)This allows the script to catch and fail on command failures (e.g., if
basenameis unavailable). However, given the script's overall robustness, this refactor can be deferred.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
scripts/opencode-analysis/README.mdis excluded by!**/README*
📒 Files selected for processing (3)
.github/workflows/opencode_analysis.yaml(1 hunks)scripts/opencode-analysis/analyze.sh(1 hunks)scripts/opencode-analysis/create-devin-sessions.sh(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.9)
.github/workflows/opencode_analysis.yaml
24-24: shellcheck reported issue in this script: SC2086:info:2:31: Double quote to prevent globbing and word splitting
(shellcheck)
🪛 Shellcheck (0.11.0)
scripts/opencode-analysis/create-devin-sessions.sh
[warning] 17-17: Declare and assign separately to avoid masking return values.
(SC2155)
[warning] 18-18: Declare and assign separately to avoid masking return values.
(SC2155)
[warning] 44-44: session_id appears unused. Verify use (or export if used externally).
(SC2034)
[warning] 44-44: Declare and assign separately to avoid masking return values.
(SC2155)
[warning] 45-45: Declare and assign separately to avoid masking return values.
(SC2155)
[warning] 65-65: Declare and assign separately to avoid masking return values.
(SC2155)
[warning] 66-66: Declare and assign separately to avoid masking return values.
(SC2155)
[warning] 71-71: Declare and assign separately to avoid masking return values.
(SC2155)
[warning] 72-72: Declare and assign separately to avoid masking return values.
(SC2155)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Devin
🔇 Additional comments (4)
.github/workflows/opencode_analysis.yaml (2)
30-35: Verify permissions and environment propagation.The
analyze.shscript is invoked withMAX_CONCURRENCYpassed explicitly at line 35, which correctly overrides the default. Ensure that the script sources or inherits all necessary environment variables (e.g.,ANTHROPIC_API_KEY,MAX_CONCURRENCY). The current setup looks correct, but confirm that OpenCode CLI can access the API key at runtime.
43-51: Clarify artifact handling forfindings.jsongeneration.The workflow runs
create-devin-sessions.sh parseat line 45 to generatefindings.json, then uploads it as an artifact at line 50. This artifact is later downloaded in thecreate-sessionsjob (line 60–63). Confirm that:
- The
parsecommand gracefully handles missing or malformed output files (e.g., ifanalyze.shfails).- If no valid findings are generated, an empty or minimal
findings.jsonis still created so the downstreamcreate-sessionsjob does not fail during artifact download.Run a manual test via
workflow_dispatchwithcreate_devin_sessions: falseto verify the artifact pipeline works end-to-end.scripts/opencode-analysis/create-devin-sessions.sh (2)
54-99: DEVIN_API_KEY validation is correctly placed in process_findings.The check at line 55 validates
DEVIN_API_KEYonly when thecreatecommand is invoked (viaprocess_findings), not at script startup. This is the correct approach—it allowsparseto run independently without requiring the API key to be set. The PR objectives' recommendation to move this check intoprocess_findings()appears to already be implemented.
101-116: Entrypoint and command dispatch look solid.The case statement cleanly separates
parseandcreateworkflows, with clear help text. The design allows the GitHub Actions workflow to invokeparseto generate findings and later invokecreateconditionally, which is a good separation of concerns.
scripts/opencode-analysis/analyze.sh
Outdated
| cd "$target_path" | ||
|
|
||
| if opencode run "$prompt" --agent plan --format json > "$output_file" 2>&1; then | ||
| echo "Completed: $target" | ||
| else | ||
| echo "Failed: $target" | ||
| echo "{\"error\": \"Analysis failed\", \"target\": \"$target\"}" > "$output_file" | ||
| fi | ||
| } |
There was a problem hiding this comment.
Separate stderr from JSON output to prevent corruption.
Line 57 redirects both stdout and stderr (2>&1) to the output file. If OpenCode fails, error messages are written to the JSON file, corrupting it. The fallback at line 61 then appends a malformed JSON structure, leaving the file in an unparseable state for downstream jq operations in create-devin-sessions.sh.
Separate stderr and stdout, and only write JSON output to the file:
- if opencode run "$prompt" --agent plan --format json > "$output_file" 2>&1; then
+ if opencode run "$prompt" --agent plan --format json > "$output_file" 2>/tmp/opencode-err-${target_name}.log; then
echo "Completed: $target"
else
echo "Failed: $target"
+ cat /tmp/opencode-err-${target_name}.log >&2
echo "{\"error\": \"Analysis failed\", \"target\": \"$target\"}" > "$output_file"
fiAlternatively, capture stderr to a separate diagnostics file or log it without mixing into the JSON output.
Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In scripts/opencode-analysis/analyze.sh around lines 55 to 63, stderr is
currently merged into the JSON output file (using 2>&1) which can corrupt the
JSON; change the opencode invocation to write only stdout (the JSON) to
"$output_file" and redirect stderr to a separate diagnostics/log file (e.g.,
"$output_file.err" or a central log) so errors aren’t mixed into the JSON; keep
the existing if/else logic but ensure the failure branch still writes the
fallback JSON to "$output_file" and optionally write a human-readable error
message or the captured stderr into the diagnostics file for debugging.
Co-Authored-By: yujonglee <yujonglee.dev@gmail.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
.github/workflows/opencode_analysis.yaml (3)
34-36: Consider pinning OpenCode CLI version. Thecurl | bashpattern installs the latest version without version control. Consider using a specific version pin or checksum to ensure reproducibility and reduce supply-chain risk.
55-62: Improve error handling for missing or invalid analysis files.Line 62 silently falls back to
{}if analysis.json is missing, which can result in API calls with empty findings. This masks OpenCode failures and may create unnecessary Devin sessions. Add explicit validation: check if the file exists and log warnings if it's missing or invalid. Consider failing the job if analysis is unavailable for a target, rather than proceeding with empty data.
63-67: Add proper error logging for Devin API calls.The
|| trueoperator on line 66 suppresses curl errors, making it hard to debug API failures. Consider capturing and logging errors, or at least checking the HTTP response code before silently continuing. Additionally, verify that the 2-second sleep between requests aligns with Devin API rate limits.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/opencode_analysis.yaml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: fmt
- GitHub Check: Redirect rules - hyprnote
- GitHub Check: Header rules - hyprnote
- GitHub Check: Pages changed - hyprnote
- GitHub Check: Devin
🔇 Additional comments (3)
.github/workflows/opencode_analysis.yaml (3)
1-11: LGTM—Triggers and inputs are well-configured. Cron schedule correctly targets Mondays 06:00 UTC, and workflow_dispatch inputs align with concurrency control and Devin session creation.
14-17: Verifyinputs.max_concurrencybehavior in scheduled runs.Line 17 uses
${{ inputs.max_concurrency || 3 }}, but theinputscontext is only available duringworkflow_dispatchtriggers. In scheduled runs, this expression may fail or use an undefined value. Confirm that the fallback to3is reliable, or refactor to use an environment variable or conditional.
44-48: LGTM—Artifact upload is correctly configured. Per-target naming ensures uniqueness, and 30-day retention is appropriate.
| - run: | | ||
| for dir in findings/*/; do | ||
| target=$(basename "$dir" | sed 's/analysis-//') | ||
| analysis=$(cat "$dir/analysis.json" 2>/dev/null || echo "{}") | ||
| curl -s -X POST "https://api.devin.ai/v1/sessions" \ | ||
| -H "Authorization: Bearer $DEVIN_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d "{\"prompt\": \"Fix issues in fastrepl/hyprnote $target based on: $analysis\"}" || true | ||
| sleep 2 | ||
| done | ||
| env: | ||
| DEVIN_API_KEY: ${{ secrets.DEVIN_API_KEY }} |
There was a problem hiding this comment.
CRITICAL: Fix JSON injection vulnerability and add DEVIN_API_KEY validation.
This script has multiple blocking issues:
-
JSON Injection (line 66): The
$analysisvariable is directly interpolated into the JSON payload without escaping. If analysis contains quotes, backslashes, or newlines, the JSON will be malformed. Example:analysis='{"error": "quote\"inside"}'will break the payload. -
Missing DEVIN_API_KEY validation (lines 59-70): Per the PR checklist, the DEVIN_API_KEY should be validated before use. Currently, if the secret is not set, curl will POST with an empty Bearer token and fail silently due to
|| true. Add an early check to halt the job if the key is missing. -
Artifact path extraction (line 61): The sed command assumes artifact names in
findings/*/, butactions/download-artifact@v4creates subdirectories likefindings/<artifact_name>/. Verify the actual structure and adjust the extraction logic accordingly.
Apply this diff to fix the issues:
- run: |
+ if [ -z "$DEVIN_API_KEY" ]; then
+ echo "ERROR: DEVIN_API_KEY is not set" >&2
+ exit 1
+ fi
for dir in findings/*/; do
target=$(basename "$dir" | sed 's/analysis-//')
- analysis=$(cat "$dir/analysis.json" 2>/dev/null || echo "{}")
- curl -s -X POST "https://api.devin.ai/v1/sessions" \
+ if [ ! -f "$dir/analysis.json" ]; then
+ echo "WARNING: $dir/analysis.json not found, skipping $target" >&2
+ continue
+ fi
+ analysis=$(cat "$dir/analysis.json")
+ payload=$(jq -n --arg prompt "Fix issues in fastrepl/hyprnote $target based on: $analysis" '{prompt: $prompt}')
+ curl -s -X POST "https://api.devin.ai/v1/sessions" \
-H "Authorization: Bearer $DEVIN_API_KEY" \
-H "Content-Type: application/json" \
- -d "{\"prompt\": \"Fix issues in fastrepl/hyprnote $target based on: $analysis\"}" || true
+ -d "$payload" || true
sleep 2
doneAdditionally, verify the artifact directory structure after download to ensure the path extraction is correct.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - run: | | |
| for dir in findings/*/; do | |
| target=$(basename "$dir" | sed 's/analysis-//') | |
| analysis=$(cat "$dir/analysis.json" 2>/dev/null || echo "{}") | |
| curl -s -X POST "https://api.devin.ai/v1/sessions" \ | |
| -H "Authorization: Bearer $DEVIN_API_KEY" \ | |
| -H "Content-Type: application/json" \ | |
| -d "{\"prompt\": \"Fix issues in fastrepl/hyprnote $target based on: $analysis\"}" || true | |
| sleep 2 | |
| done | |
| env: | |
| DEVIN_API_KEY: ${{ secrets.DEVIN_API_KEY }} | |
| - run: | | |
| if [ -z "$DEVIN_API_KEY" ]; then | |
| echo "ERROR: DEVIN_API_KEY is not set" >&2 | |
| exit 1 | |
| fi | |
| for dir in findings/*/; do | |
| target=$(basename "$dir" | sed 's/analysis-//') | |
| if [ ! -f "$dir/analysis.json" ]; then | |
| echo "WARNING: $dir/analysis.json not found, skipping $target" >&2 | |
| continue | |
| fi | |
| analysis=$(cat "$dir/analysis.json") | |
| payload=$(jq -n --arg prompt "Fix issues in fastrepl/hyprnote $target based on: $analysis" '{prompt: $prompt}') | |
| curl -s -X POST "https://api.devin.ai/v1/sessions" \ | |
| -H "Authorization: Bearer $DEVIN_API_KEY" \ | |
| -H "Content-Type: application/json" \ | |
| -d "$payload" || true | |
| sleep 2 | |
| done | |
| env: | |
| DEVIN_API_KEY: ${{ secrets.DEVIN_API_KEY }} |
Summary
Adds an automated code analysis workflow using OpenCode CLI with Claude AI to periodically scan the codebase for bugs, refactoring opportunities, and security concerns. Findings can optionally trigger Devin sessions to create fix PRs.
The workflow runs weekly (Mondays 6 AM UTC) via cron or can be triggered manually. It analyzes 11 key components (plugins, crates, apps) with max 3 concurrent analyses using the OpenCode
planagent (read-only, no code changes).Updates since last revision
Simplified to a single workflow file with all logic inline (no separate scripts or README).
Review & Testing Checklist for Human
ANTHROPIC_API_KEYbut OpenCode docs show Claude Max uses OAuth via/connect. Test if API key auth works in CI or if a different auth method is needed.workflow_dispatchbefore relying on cron schedule. The OpenCode CLI installation and execution in CI is untested.plugins/listener) which become artifact names likeanalysis-plugins/listener. Verify this doesn't cause upload issues.opencode run --format jsonoutputs parseable JSON. This should be validated.Recommended test plan:
ANTHROPIC_API_KEYsecret to repocreate_devin_sessions: falseNotes
Requested by: yujonglee (@yujonglee)
Devin session: https://app.devin.ai/sessions/97107827f74345a5ab430b1e576218f2