-
Notifications
You must be signed in to change notification settings - Fork 51
Optimize webhooks JSON with compact structure and string interning #232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
bfb514f to
4dacd2c
Compare
4dacd2c to
27aec5d
Compare
| run: cd languageservice && npm test -- --testPathPattern=eventPayloads | ||
| - name: Verify validation tests ran | ||
| run: | | ||
| if [ ! -f languageservice/src/context-providers/events/webhooks.full.validation-complete ]; then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the test is normally skipped when the full webhooks file doesn't exist, I made the test write a marker file. With the marker file, we can be sure the test actually ran.
The full webhooks file (unoptimized) is written above by the script npm run update-webhooks
| exit 1 | ||
| fi | ||
| validate-webhooks: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this job to validate the optimized webhooks JSON is equivalent to the full webhooks JSON file.
Refer changes to the file eventPayloads.test.ts in this PR
27aec5d to
8fbd093
Compare
| @@ -1,310 +0,0 @@ | |||
| import {promises as fs} from "fs"; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I renamed this file to update-webhooks.ts but there were also significant updates anyway
| @@ -1,7 +1,8 @@ | |||
| import {data, DescriptionDictionary} from "@actions/expressions"; | |||
|
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
eventsPayloads.ts reads the compact webhook JSON files which contains the keys and descriptions for each GitHub event payload. This information is used for github.event.* syntax validation, autocompletion, and hover descriptions.
A unit test that runs during CI validates the compact webhook JSON files are parsed into a form that exactly matches the full unoptimized webhooks JSON.
The script update-webhooks.ts downloads the full webhook JSON from the GitHub REST API description repository:
import schemaImport from "rest-api-description/descriptions/api.github.com/dereferenced/api.github.com.deref.json";10dd69f to
2b6d46c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR optimizes webhook payload JSON files used for GitHub Actions autocompletion and hover documentation by introducing a compact array format, object deduplication, and string interning. The optimizations achieve a 55% reduction in minified file size (464 KB → 209 KB) and 65% reduction when gzipped (63 KB → 22 KB).
Key Changes:
- Introduced compact array format for params using type-based dispatch instead of verbose objects
- Implemented string interning to deduplicate 414 property names into a shared
strings.jsonfile - Extracted event filtering configuration from TypeScript to
event-filters.jsonfor use in both generation and test code
Reviewed changes
Copilot reviewed 11 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
languageservice/src/context-providers/events/strings.json |
New string table containing 414 interned property names referenced by index |
languageservice/src/context-providers/events/eventPayloads.ts |
Updated to load separate string table and decode compact format with string interning support |
languageservice/src/context-providers/events/eventPayloads.test.ts |
Added comprehensive validation tests comparing optimized output against full source |
languageservice/src/context-providers/events/event-filters.json |
New JSON file defining dropped/kept events, replacing hardcoded TypeScript constants |
languageservice/script/webhooks/update-webhooks.ts |
New generation script implementing compact format conversion, deduplication, and string interning |
languageservice/script/webhooks/event-filters.ts |
TypeScript constants for event filtering (note: appears unused, actual imports use JSON file) |
languageservice/script/webhooks/deduplicate.ts |
Updated to handle both compact and legacy param formats with helper functions |
languageservice/package.json |
Updated scripts to include new strings.json in minification and point to new update-webhooks script |
docs/json-data-files.md |
Comprehensive documentation of compact format, string interning, and optimization details |
.gitignore |
Updated patterns for new full.json and validation-complete marker files |
.github/workflows/buildtest.yml |
Added separate validate-webhooks job to verify optimization correctness |
languageservice/script/webhooks/index.ts |
Removed (replaced by update-webhooks.ts) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d953314 to
c034e38
Compare
- Convert params to compact array format (type-based dispatch) - Intern duplicate property names into string table - Use negative indices for object references to distinguish from string indices - Rename objects.json → webhooks.objects.json, add webhooks.strings.json - Move event filters to JSON for maintainability - Add CI validation job to verify optimization correctness Reduces combined minified size by ~67% (453 KB → 148 KB), ~27% gzipped (23 KB → 17 KB).
c034e38 to
c04c1b2
Compare
Overview
This PR optimizes the webhook payload JSON files used for GitHub Actions autocompletion and hover documentation.
Total JSON payload (gzipped): 42 KB → 36 KB (15% reduction)
Changes
New Optimizations
Compact array format — Params converted from verbose objects to type-dispatched arrays:
String interning — Duplicate property names stored in
webhooks.strings.jsonand referenced by non-negative indexNegative indices for objects — Object references now use negative numbers (
-(index + 1)) to distinguish from string indices, allowing all duplicate names to be internedFile Renames
objects.json→webhooks.objects.jsonwebhooks.strings.jsonfor interned property namesOther Changes
event-filters.jsonfor maintainability"//"comment key towebhooks.jsonpointing to documentationvalidate-webhooks) to verify optimization correctnessSize Reduction
Testing
webhooks.full.json) to ensure no data lossDocumentation
Updated
docs/json-data-files.mdwith: