Skip to content

ci: add nightly contract tests with long-running tests and Slack notification#136

Draft
devin-ai-integration[bot] wants to merge 4 commits intomainfrom
devin/1770832864-nightly-contract-tests
Draft

ci: add nightly contract tests with long-running tests and Slack notification#136
devin-ai-integration[bot] wants to merge 4 commits intomainfrom
devin/1770832864-nightly-contract-tests

Conversation

@devin-ai-integration
Copy link
Contributor

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

N/A

Describe the solution you've provided

Adds a new GitHub Actions workflow that runs the Java server SDK contract tests nightly (4am UTC) with the -enable-long-running-tests flag enabled. If the tests fail, a Slack notification is sent to alert the team.

Key details:

  • Schedule: Runs daily at 4am UTC via cron, with workflow_dispatch for manual triggering
  • Test execution: Reuses the existing ci composite action for build/setup, then runs the contract test make targets with TEST_HARNESS_PARAMS="-enable-long-running-tests"
  • Slack notification: Uses slackapi/slack-github-action with an incoming webhook (SLACK_WEBHOOK_URL secret) to post to Slack on failure, including a link to the failed run

Items for reviewer attention:

  1. SLACK_WEBHOOK_URL secret — this secret must be configured in the repo settings and the webhook must be routed to #sdks-java. The workflow itself does not specify a channel; the channel is determined by the webhook configuration.
  2. No timeout-minutes — long-running tests could potentially hang indefinitely. Consider adding a timeout.
  3. Notification scope — the Slack job triggers on failure only, not cancelled or timed_out. Decide if those states should also notify.
  4. Contract test steps are split out rather than using the existing contract-tests composite action, because TEST_HARNESS_PARAMS needs to be passed to the run-contract-tests target specifically. Verify this is acceptable.

Describe alternatives you've considered

  • Could modify the existing contract-tests composite action to accept a test_harness_params input, but that would change the shared action's interface for a nightly-only concern.
  • Could use rtCamp/action-slack-notify (used by ldcli) instead of slackapi/slack-github-action, but the official Slack action is more widely used across LD repos (terraform-provider-launchdarkly, streamer, ld-docs-private).

Additional context

Link to Devin run
Requested by: tanderson@launchdarkly.com

…fication

Co-Authored-By: tanderson@launchdarkly.com <tanderson@launchdarkly.com>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

tanderson-ld and others added 3 commits February 11, 2026 15:54
Co-authored-by: semgrep-code-launchdarkly[bot] <167133144+semgrep-code-launchdarkly[bot]@users.noreply.github.com>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reviewers: These tests are not passing because they are written against the Go impl. I need to update the FDv2 Data System spec to specify the edge case behavior and timings to either match Go or update Go and then update the Java, Dotnet, and Node impls to match that.

if: ${{ inputs.test_slack_notification == true || inputs.test_slack_notification == 'true' }}
steps:
- name: Send test Slack notification
uses: slackapi/slack-github-action@v2.1.0

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semgrep identified an issue in your code:
An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

To resolve this comment:

✨ Commit Assistant fix suggestion

Suggested change
uses: slackapi/slack-github-action@v2.1.0
uses: slackapi/slack-github-action@3f4fd9c4819a1c06c6f2d56a8dd2c19b3bd33080
View step-by-step instructions
  1. Replace uses: slackapi/slack-github-action@v2.1.0 with a reference to the action's full commit SHA instead of the version tag.
  2. Visit the action's repository at https://github.com/slackapi/slack-github-action, go to the "Releases" or "Tags" section, and find the commit SHA corresponding to v2.1.0.
  3. Use the full 40-character commit SHA in the uses field, for example: uses: slackapi/slack-github-action@b90c43eb73a2ada7ef1e8c9a23e4bab654742b91 (replace with the correct SHA for v2.1.0).
  4. Make this change for all instances of slackapi/slack-github-action@v2.1.0 in your workflow files.
    Using the full commit SHA ensures the GitHub Action source cannot change unexpectedly.
💬 Ignore this finding

Reply with Semgrep commands to ignore this finding.

  • /fp <comment> for false positive
  • /ar <comment> for acceptable risk
  • /other <comment> for all other reasons

Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by third-party-action-not-pinned-to-commit-sha.

You can view more details about this finding in the Semgrep AppSec Platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant