2026年5月28日 (周四)
Agentic AI正在打击困难的部分:现实的任务,现实的绳索,以及可靠的测量. 新的基准表明,我们还没有进入 " 手动企业自动化 " 阶段,新的培训框架正在试图通过从真正的代理工具中捕捉到具有象征意义的轨迹来缩小这一差距。 实际的外卖是先投资于evals和仪器,并将光滑剂演示作为假说而非证明.
Agentic AI正在打击困难的部分:现实的任务,现实的绳索,以及可靠的测量. 新的基准表明,我们还没有进入 " 手动企业自动化 " 阶段,新的培训框架正在试图通过从真正的代理工具中捕捉到具有象征意义的轨迹来缩小这一差距。 实际的外卖是先投资于evals和仪器,并将光滑剂演示作为假说而非证明.
ITBench-AA发现代理企业信息技术任务的前沿模型仍然低于50%
Hugging Face发布IT Bench-AA(通过人工分析和IBM),将其定位为第一个专注于代理企业IT任务的基准,据报道前沿模型得分低于50%.
企业IT工作充满了不便的限制(许可,变更窗口,票务工作流程,部分信息). 如果顶级模型无法在一个基准中连贯地完成这些任务,团队应当期望生产过程中的高度差异和隐藏的集成成本.
- 01 Enterprise IT tasks stress different failure modes than coding puzzles: state tracking, policy adherence, tool execution, and recovery from partial failures.
- 02 A sub-50% headline is a reminder that ‘agentic’ does not automatically mean ‘reliable’. You need guardrails, approvals, and fallbacks for real operations.
- 03 Benchmarks like this are most useful when you map them to your own workflows, then add task-specific acceptance tests and incident playbooks.
If you are evaluating agents for internal IT automation, build a small ‘shadow benchmark’ from your last 20 real tickets (sanitized): include access failures, ambiguous requests, and multi-step approvals. Score agents on completion, time-to-rollback, and policy compliance, not just whether they reached an endpoint. Treat any task that can impact production as ‘human-in-the-loop by default’ until you have measured stability over weeks.
NVIDIA 的极地捕捉到象征真实的轨迹,
MarkTechPost总结了NVIDIA的极地,这是一个推出框架,在代理吊带和推论服务器之间插入一个模型API代理,以捕捉令牌级别的相互作用,并在不改变吊带的情况下重建GRPO的训练轨迹.
代理人培训方面的一个巨大差距是,在如何对代理人进行实际利用的评价与如何为培训收集数据之间不匹配。 如果极地的方法被概括,那么在保持同样的生产控制、工具化和UI循环的同时,可以更容易地改进物剂。
- 01 Harness realism matters. Training on synthetic transcripts can miss the exact token-level control flow that production harnesses induce.
- 02 A proxy-based approach can reduce engineering friction by avoiding invasive changes to the agent runtime while still producing trainer-ready data.
- 03 Reported gains are harness-dependent, which is the point: agent performance can be highly sensitive to the surrounding harness and tool surface.
If you run a coding-agent harness (or any tool-augmented agent loop), instrument it like a product: log every model request/response, tool call, tool output, and final user-visible action with a stable trace id. Even if you do not do RL training, this gives you reproducible failure cases and lets you compare versions. If you do plan RL, ensure your logging preserves token boundaries and tool I/O exactly, or you will train on distorted trajectories.
Meta扩展了Instagram、Facebook和WhatsApp的付费订阅,
TechCrunch Reports Meta推出全球主要消费应用的付费订阅,
订阅会改变产品的奖励:它们可以减少对只广告货币化的依赖,并创建一个直接路径来捆绑AI特性. 对于用户和企业来说,它提出了什么是付费墙(支持、核实、分发)以及如何包装AI工具的问题。
- 01 Paid tiers can become the delivery vehicle for AI features (and for feature gating) even in apps that were historically free-to-use.
- 02 Bundling across apps increases lock-in and can reshape creator and SMB workflows if AI tools are tied to subscription identity and support tiers.
- 03 For teams building on these platforms, product changes can be sudden. Expect shifting APIs, policy constraints, and pricing experiments around AI.
If your business depends on Meta surfaces (ads, creators, messaging), prepare for subscription-driven segmentation: list the critical workflows (support, verification, messaging volume, moderation, analytics), then track which ones move into paid tiers. Budget for experimentation, and avoid coupling core operations to any single ‘AI add-on’ until pricing and policy stabilize.
EAGLE 3.1 旨在稳定生产推断中的投机解码
MarkTechPost强调EAGLE 3.1是一种投机性的解码更新,旨在解决实际部署中的不稳定性和注意力漂移问题.
文件研究:生产计量偏差
arXiv论文认为,共同的客户端基准设计可以大规模扭曲延迟和吞吐量的测量。