Skip to content

feat(perplexity): native content-block streaming events#38070

Open
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-perplexity
Open

feat(perplexity): native content-block streaming events#38070
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-perplexity

Conversation

@nick-hollon-lc

Copy link
Copy Markdown
Contributor

ChatPerplexity now implements a native stream_events(version="v3") path (_stream_chat_model_events / _astream_chat_model_events) that builds text and tool_call content blocks directly from Perplexity's streaming delta, rather than riding the compat bridge.

The converter is bespoke and adds no langchain-openai dependency (mirroring the groq approach). It reuses the existing _create_usage_metadata. Perplexity reports cumulative token usage per chunk, so the message total is taken from the last chunk that carries usage rather than accumulated across chunks.

Perplexity's search extras (citations, search_results, images, related_questions, videos, reasoning_steps) are surfaced on the assembled message's response_metadata. The v3 assembled message has no additional_kwargs channel, and the compat bridge drops additional_kwargs entirely — so this native path preserves search data that the bridge-based v3 path would otherwise lose. Reasoning text arrives inline (Perplexity emits <think>…</think> within content, with no separate field), so it surfaces as ordinary text.

The public stream_events contract is unchanged and the compat bridge remains the fallback. Surfacing the search extras as dedicated non-standard content blocks was intentionally deferred to keep this change low-risk; it can be revisited.

Reviewer note: unit tests patch the SDK at the chat.completions.create boundary (no network) and assert the cumulative-usage "last total wins" behavior plus the round-trip of citations/model_name onto the assembled message's response_metadata.

@github-actions github-actions Bot added feature For PRs that implement a new feature; NOT A FEATURE REQUEST integration PR made that is related to a provider partner package integration internal perplexity `langchain-perplexity` package issues & PRs size: L 500-999 LOC labels Jun 11, 2026

@open-swe open-swe Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open SWE Review found 1 potential issue.

Open in WebView Open SWE trace

Comment on lines +131 to +132
if len(choices) == 0:
continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Empty-choice streams lose usage events

This continue means the native v3 path never emits message-start or message-finish if Perplexity returns only choices: [] chunks (for example an empty completion that still carries usage/metadata). Before this native hook existed, stream_events(version="v3") went through _stream, whose empty-choice branch intentionally yielded an empty chunk so the stream was not empty and downstream callers still received usage metadata. With the hook installed, the same response now produces no events and loses the final usage; the async converter has the same branch.

(Refers to lines 131-132)


Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.

@nick-hollon-lc Nick Hollon (nick-hollon-lc) force-pushed the nh/native-content-block-streaming-perplexity branch from ad9d325 to 9f01e8c Compare June 11, 2026 15:23

@open-swe open-swe Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open SWE Review found 1 potential issue.

Open in WebView Open SWE trace

Comment on lines +1398 to +1400
raw = self.client.chat.completions.create(
messages=message_dicts, stream=True, **params
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 V3 events bypass Responses routing

The native stream_events(version="v3") hook always calls Chat Completions here instead of applying the new _use_responses_api / _to_responses_payload routing used by _stream and _generate. With ChatPerplexity(use_responses_api=True).stream_events(..., version="v3"), or the documented auto-routing case stream_events(..., tools=[{"type": "web_search"}], version="v3"), core selects this native hook and sends Responses-only fields/built-in tools to chat.completions.create, which this PR's own routing logic says Chat Completions rejects. The async hook has the same direct Chat Completions call.

(Refers to lines 1398-1400)


Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Perplexity native stream converter documents important provider quirks upfront: cumulative usage on the last chunk, inline redacted_thinking as text, and citations moved to response_metadata on the v3 path. Using BlockStreamTracker keeps this consistent with other partner streaming PRs in the series.

@mdrxy

Copy link
Copy Markdown
Member

Mohamed-Taqy Salmi (@mohamedtaqysalmi) stop posting unsolicited comments, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature For PRs that implement a new feature; NOT A FEATURE REQUEST integration PR made that is related to a provider partner package integration internal perplexity `langchain-perplexity` package issues & PRs size: L 500-999 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants