fix(celery): Get driver_type from broker's URL#5461
Draft
ericapisani wants to merge 4 commits intomasterfrom
Draft
fix(celery): Get driver_type from broker's URL#5461ericapisani wants to merge 4 commits intomasterfrom
driver_type from broker's URL#5461ericapisani wants to merge 4 commits intomasterfrom
Conversation
Under the hood within Kombu (the messaging library used by Celery), the `driver_type` is ultimately derived from the broker's URL that's provided in the Celery configuration by the user via either the `broker` or `broker_url` arguments. We can accomplish the same thing in our context, removing the need to open a connection to the broker to determine the `driver_type` to set in the span. Fixes PY-2071 Fixes #5428
Contributor
Semver Impact of This PR🟢 Patch (bug fixes) 📋 Changelog PreviewThis is how your changes will appear in the changelog. Bug Fixes 🐛Celery
Openai Agents
Documentation 📚
Internal Changes 🔧Openai Agents
Other
🤖 This preview updates automatically when you update the PR. |
Contributor
Codecov Results 📊✅ 13 passed | Total: 13 | Pass Rate: 100% | Execution Time: 6.23s 📊 Comparison with Base Branch
All tests are passing successfully. ✅ Patch coverage is 100.00%. Project has 13698 uncovered lines. Files with missing lines (180)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
+ Coverage 25.71% 30.82% +5.11%
==========================================
Files 189 189 —
Lines 19795 19801 +6
Branches 6408 6408 —
==========================================
+ Hits 5090 6103 +1013
- Misses 14705 13698 -1007
- Partials 431 467 +36Generated by Codecov Action |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Uses a subset of mapping that Kombu uses, along with the default `driver_type` value that it uses when Kombu hasn't explicitly set one for a given Transport class (like SLMQ).
…getsentry/sentry-python into py-2071-get-driver-type-from-broker-url
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 3 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
|
|
||
| # "N/A" is set as the fallback value in Kombu when a `driver_type` | ||
| # is not set on the Kombu Transport class itself. | ||
| return url_scheme_to_driver_type_mapping.get(scheme, "N/A") |
There was a problem hiding this comment.
URL schemes with + modifier not handled correctly
Medium Severity
Kombu supports URL schemes with + modifiers like redis+socket:///path/to/socket for Redis over Unix sockets. Python's urlparse returns the full scheme (e.g., "redis+socket"), which doesn't match any key in url_scheme_to_driver_type_mapping, causing _get_driver_type_from_url_scheme to incorrectly return "N/A" instead of "redis". The old code using conn.transport.driver_type correctly resolved these to the base transport's driver_type. The scheme needs to be split on + and the base part used for lookup.
Additional Locations (1)
| "memory": "memory", | ||
| "redis": "redis", | ||
| "rediss": "redis", | ||
| "SQS": "sqs", |
There was a problem hiding this comment.
Uppercase SQS mapping key is unreachable dead code
Low Severity
The "SQS" (uppercase) key in url_scheme_to_driver_type_mapping can never be matched. Python's urlparse normalizes schemes to lowercase per RFC 3986, so urlparse("SQS://...").scheme returns "sqs". The lowercase "sqs" key already handles this correctly, making the uppercase entry dead code that may mislead future maintainers.
| "etcd": "etcd", | ||
| "pyro": "pyro", | ||
| "gcpubsub": "gcpubsub", | ||
| } |
There was a problem hiding this comment.
Mapping missing several valid Kombu transport URL schemes
Medium Severity
The url_scheme_to_driver_type_mapping is incomplete compared to the Kombu TRANSPORT_ALIASES it references in the source comment. It's missing at least pyamqp (→ "amqp"), librabbitmq (→ "amqp"), azureservicebus (→ "azureservicebus"), and azurestoragequeues (→ "azurestoragequeues"). Users with these broker URL schemes will get "N/A" instead of the correct driver_type, which is a regression from the old conn.transport.driver_type approach.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Under the hood within Kombu (the messaging library used by Celery), the
driver_typeis ultimately derived from the broker's URL that's provided in the Celery configuration by the user via either thebrokerorbroker_urlarguments.We can accomplish the same thing in our context, removing the need to open a connection to the broker to determine the
driver_typeto set in the span.The
broker_urlis set in both of the following scenarios:app = Celery("tasks", broker_url=<your-broker-url>)app = Celery("tasks", broker=<your-broker-url>)Issues
driver_typefrom elsewhere in Celery tasks #5428Helpful links for context
driver_typevalues in kombu