> Sky is a powerful natural language interface for the Mac. With Sky, AI works alongside you, whether you’re writing, planning, coding, or managing your day. Sky understands what’s on your screen and can take action using your apps.
> We will bring Sky’s deep macOS integration and product craft into ChatGPT, and all members of the team will join OpenAI.
AI browsing the web is dumb AF if you think about it. Using an API through a REPL is so much better, we're doing all this work to basically work around jackass site operators who make everything require javascript and don't provide a documented user facing API.
The irony is that as the agentic boom really takes off, all these no-api, no accessibility sites are going to lose to small competitors who just offer a reliable agent interface, so people can use their service without having to use their service. Good riddance to the dinosaurs.
Obviously an API is better but realistically we aren't going to convince every web service to offer an API overnight and people want to be able to e.g. make reservations through chatgpt today.
Yep, the only way to convince companies to offer an API is to implement an agent that slowly but surely works around whatever trainwreck of a web experience they put in its way and then give them an option to make it smoother by offering an AI.
See: mobile websites. They sucked so badly that "desktop internet, not mobile internet" was a big selling point of the original iPhone. Then, once mobile had enough market share to "set the terms," we went back to having special mobile versions (or even mobile-first), but this time it didn't suck. Part of that was tech, but most of it was mobile acquiring a critical mass of marketshare, and the winner of the mobile wars won using an all-important temporary workaround stepping stone that solved the chicken-egg problem.
But maybe if you look from a first principles standpoint, do most human tasks decompose to some form of these same 4-6 tasks? (not talking about brainstorming, which is already well covered, or socializing, which is offline)
The only useful case I can think of is if you’re on a website with a big unstructured list or collection and you want to filter or reformat the data. For example, say you’re looking at a listing of houses for sale and you want to see only the ones that are painted blue, but the site doesn’t have that kind of structured data. Then AI could help by looking at the images and picking those out. Still, that’s probably not a very common situation, and you could do something similar with a bit of scripting and feeding that data into an AI manually. But for people who don’t know how to code, or are intimidated by it even when AI writes it for them, I guess it could be useful.
Oh and maybe one more thing to just give you the content that you're looking for like on all of these recipe sites with walls of text and images for SEO purposes where you just want the recipe. I guess that could be useful to just ask show me the recipe.
The demo looks like holding a robot's hand while they do something that would normally take me 15 seconds anyway. I have mostly found AI to be useful for search/research, not creating a middle-man between my friends and myself who has the "feature" of knowing what the star ratings on Google Maps imply.
That’s a weird claim. Sky never shipped anything, and they’ve been building for 2 years. You don’t think Apple has any internal demos that are comparable? How do you know?
You didn't read the link, did you? Sky was written by the people who wrote Shortcuts, and Apple not only failed to retain that talent but also make their App Intents (and previous automation features) usable.
I did read the link. The people working on Sky were exactly the people tasked with making App Intents usable, so again it doesn’t make sense why you give a bunch of credence to semi-public demoware in one context versus demoware you don’t have access to. Ari worked at Apple for a decade and Shortcuts went nowhere. Then he made a demo that was acquired by a company that’s buying a lot of teams with good demos. What’s the huge miss, nothing has actually shipped.
Whats the big deal? If the management at Apple thought they were important and critical to Apples future, they wouldve made it work. The reality is, they dont believe this to be the case.
