Categories Machine Learning

xAI Introduced Agentic Server-Side Tool Calling

Step by step

The model evaluates the user’s question and existing context to identify gaps in knowledge.

It decides whether to generate a final answer or invoke a tool (for example, web search, X post search, or code execution).

If needed, it picks the right tool and parameters, then executes it in real-time on the server.

Results are fed back into the model’s reasoning, and the loop repeats until the agent has enough info for a comprehensive response.

Then you get the final answer, plus optional real-time streaming updates and citations for traceability.

There is real-time web search/browsing, X (Twitter) searches (keyword, semantic, user, threads), Python code execution and viewing X videos or images.

In streaming mode, you can watch tool calls unfold via response chunks, but tool outputs stay internal to the agent.

Offloading

Of course there is the challenge of offloading functionality and autonomy to the model, as apposed to building a solution with a framework like LangChain.

Leveraging the built-in orchestration of xAI makes for quick deployment.

But for an enterprise implementation where granular and fine-grained tweaking is required, a framework will be needed where the functionality is more decomposed.

So one will have to consider the trade-offs between server-side offloading and more granular framework management.

Code Generation

Something I find interesting, is that the the code_execution tool is a good example of how a model like Grok can autonomously generate code to achieve a specific goal, embodying what’s often called agentic code generation or self-programming in AI systems.

Here is a working code example…

Install the xai-sdk

%pip install xai-sdk

Set your xAI API key as an environment variable…

os.environ['XAI_API_KEY'] = 'Your XAI API Key'

Below is the program you can run by copying and pasting it into a Colab notebook…

You can see the three tools which are available to the model (grok-4-fast) are

  • Web Search
  • Searching X
  • Code Execution.
import os

from xai_sdk import Client
from xai_sdk.chat import user
from xai_sdk.tools import web_search, x_search, code_execution

client = Client(api_key=os.getenv("XAI_API_KEY"))
chat = client.chat.create(
model="grok-4-fast", # reasoning model
# All server-side tools active
tools=[
web_search(),
x_search(),
code_execution(),
],
)

# Feel free to change the query here to a question of your liking
chat.append(user("What are the latest updates from xAI?"))

is_thinking = True
for response, chunk in chat.stream():
# View the server-side tool calls as they are being made in real-time
for tool_call in chunk.tool_calls:
print(f"nCalling tool: {tool_call.function.name} with arguments: {tool_call.function.arguments}")
if response.usage.reasoning_tokens and is_thinking:
print(f"rThinking... ({response.usage.reasoning_tokens} tokens)", end="", flush=True)
if chunk.content and is_thinking:
print("nnFinal Response:")
is_thinking = False
if chunk.content and not is_thinking:
print(chunk.content, end="", flush=True)

print("nnCitations:")
print(response.citations)
print("nnUsage:")
print(response.usage)
print(response.server_side_tool_usage)
print("nnServer Side Tool Calls:")
print(response.tool_calls)

And the output where the x_keyword_search tool is invoked.

Thinking... (306 tokens)
Calling tool: x_keyword_search with arguments: {"query":"from:xai","limit":10,"mode":"Latest"}
Thinking... (710 tokens)

Final Response:
### Latest Updates from xAI (as of October 18, 2025)

Based on recent posts from xAI's official X account (@xai), here are the most recent key updates. xAI focuses on advancing AI through models like Grok and tools like Imagine for creative generation. I'll highlight the top developments in chronological order (most recent first):

#### 1. **October 11, 2025: Ani Enters "TRON Mode" in Grok App**
- xAI announced that "Ani" (a character or feature within Grok's AI ecosystem, often used in video and animation demos) is now in "TRON mode." This appears to be a fun, thematic update inspired by the TRON franchise, enabling grid-like digital experiences.
- Users are encouraged to update the Grok app to "enter the grid."
- Includes a demo video showcasing this mode.
- This ties into ongoing enhancements for immersive, interactive AI experiences.

#### 2. **October 7, 2025: Launch of Imagine v0.9 – Advanced Video Generation Model**
- xAI released **Imagine v0.9**, a major upgrade to their native audio + video generation tool. It's a significant leap from v0.1, with improvements in:
- **Visual quality and motion**: Fluid, realistic movements with higher precision.
- **Audio integration**: Natural dialogue, expressive singing with clear vocals, and synchronized sound (e.g., immersive audio for scenes like a dragon breathing fire).
- **Dynamic effects**: Intelligent camera shifts, focus changes, and rhythm-based animations (e.g., smooth dance moves).
- **Cinematic output**: Generates complete videos with audio-visual harmony—no post-editing needed.
- Now available **for free** across all xAI products at [grok.com/imagine](https://grok.com/imagine).
- Examples shared include:
- A dragon video with synced sound (generated by user @GlitterPixely).
- Ani demonstrating singing, dancing, and motion.
- xAI is actively seeking user feedback on X to inform future upgrades. The team emphasized incorporating community input for continuous improvements.

#### 3. **September 25, 2025: Expansion of xAI for Government**
- xAI launched an initiative to provide **Frontier AI access** (including Grok 4 and Grok 4 Fast models) to U.S. Federal Government agencies.
- Pricing: $0.42 per department for 18 months (starting immediately).
- Dedicated team of Grok engineers to support government use cases.
- They're hiring mission-driven engineers to expand the team—check job listings on X for opportunities.
- This aims to make advanced AI more accessible for public sector applications.

These updates reflect xAI's push toward more creative, accessible, and high-fidelity AI tools, with a focus on video generation and real-world integrations. For the absolute latest, I recommend checking [x.ai](https://x.ai) or following @xai on X, as developments move quickly. If you have a specific aspect (e.g., Grok updates or technical details), let me know!

Citations:
['https://x.com/i/status/1975608132530544900', 'https://x.com/i/status/1975608122845896765', 'https://x.com/i/status/1975608017396867282', 'https://x.com/i/status/1975607901571199086', 'https://x.com/i/status/1975608070245175592', 'https://x.com/i/status/1975608007250829383', 'https://x.com/i/status/1977121515587223679', 'https://x.com/i/status/1971245659660718431', 'https://x.com/i/status/1975607953391755740', 'https://x.com/i/status/1975608184711880816']

Usage:
completion_tokens: 699
prompt_tokens: 7847
total_tokens: 9256
prompt_text_tokens: 7847
reasoning_tokens: 710
cached_prompt_text_tokens: 3773
server_side_tools_used: SERVER_SIDE_TOOL_X_SEARCH

{'SERVER_SIDE_TOOL_X_SEARCH': 1}

Server Side Tool Calls:
[function {
name: "x_keyword_search"
arguments: "{"query":"from:xai","limit":10,"mode":"Latest"}"
}
]

And seeing which tool was called:

response.tool_calls

And the output…

[function {
name: "x_keyword_search"
arguments: "{"query":"from:xai","limit":10,"mode":"Latest"}"
}]

I wanted to “force” the use of the coding tool…hence I used the query below…

chat.append(user("Solve the quadratic equation x² - 5x + 6 = 0 and plot the roots."))

With the output below:

Thinking... (445 tokens)
Calling tool: code_execution with arguments: {"code":"import sympy as spnnx = sp.symbols('x')neq = x**2 - 5*x + 6nroots = sp.solve(eq, x)nprint("Roots:", roots)nn# For plotting roots on number line, simple text plotnimport numpy as npnmin_x, max_x = 0, 5nline = [' ' * (max_x - min_x + 1) for _ in range(1)]npositions = {2: '|', 3: '|'}nnprint("\nNumber line plot of roots:")nprint('0' + ' ' * (max_x) + '5')nfor i in range(min_x, max_x + 1):n char = positions.get(i, '-')n print(char, end='')nprint("\nRoots at: 2 and 3")"}
Thinking... (489 tokens)

Final Response:
### Solution to the Quadratic Equation

The quadratic equation is ( x^2 - 5x + 6 = 0 ).

#### Factoring and Roots
This factors as ( (x - 2)(x - 3) = 0 ), so the roots are:
- ( x = 2 )
- ( x = 3 )

(Alternatively, using the quadratic formula: ( x = frac{5 pm sqrt{25 - 24}}{2} = frac{5 pm 1}{2} ), yielding the same roots.)

#### Plot of the Roots
The roots are real and lie on the number line. Here's a simple text-based visualization of the roots on a number line (scaled from 0 to 5 for clarity):

```
0 5
--||--
```
- The `|` marks indicate the roots at positions 2 and 3.
- Both roots are positive real numbers, separated by 1 unit.

Citations:
[]

Usage:
completion_tokens: 395
prompt_tokens: 4486
total_tokens: 5370
prompt_text_tokens: 4486
reasoning_tokens: 489
cached_prompt_text_tokens: 4070
server_side_tools_used: SERVER_SIDE_TOOL_CODE_EXECUTION

{'SERVER_SIDE_TOOL_CODE_EXECUTION': 1}

Server Side Tool Calls:
[function {
name: "code_execution"
arguments: "{"code":"import sympy as sp\n\nx = sp.symbols('x')\neq = x**2 - 5*x + 6\nroots = sp.solve(eq, x)\nprint(\"Roots:\", roots)\n\n# For plotting roots on number line, simple text plot\nimport numpy as np\nmin_x, max_x = 0, 5\nline = [' ' * (max_x - min_x + 1) for _ in range(1)]\npositions = {2: '|', 3: '|'}\n\nprint(\"\\nNumber line plot of roots:\")\nprint('0' + ' ' * (max_x) + '5')\nfor i in range(min_x, max_x + 1):\n char = positions.get(i, '-')\n print(char, end='')\nprint(\"\\nRoots at: 2 and 3\")"}"
}
]

Again seeing which tool was called:

response.tool_calls

And the output…

[function {
name: "code_execution"
arguments: "{"code":"import sympy as sp\n\nx = sp.symbols('x')\neq = x**2 - 5*x + 6\nroots = sp.solve(eq, x)\nprint(\"Roots:\", roots)\n\n# For plotting roots on number line, simple text plot\nimport numpy as np\nmin_x, max_x = 0, 5\nline = [' ' * (max_x - min_x + 1) for _ in range(1)]\npositions = {2: '|', 3: '|'}\n\nprint(\"\\nNumber line plot of roots:\")\nprint('0' + ' ' * (max_x) + '5')\nfor i in range(min_x, max_x + 1):\n char = positions.get(i, '-')\n print(char, end='')\nprint(\"\\nRoots at: 2 and 3\")"}"
}]

Finally

Maybe it is just my perception, but xAI is building so much functionality in the background without the fanfare that a organisation like OpenAI has.

The documentation and developer tools of xAI is expanding rapidly and as I have mentioned earlier, xAI has the distinct advantage of leveraging x (Twitter) as a powerful source of real-time data.

Press enter or click to view image in full size

Chief Evangelist @ Kore.ai | I’m passionate about exploring the intersection of AI and language. Language Models, AI Agents, Agentic Apps, Dev Frameworks & Data-Driven Tools shaping tomorrow.

Press enter or click to view image in full size

https://docs.x.ai/docs/guides/tools/overview