C has macros too, but it's a second preprocessor language. They both accomplish metaprogramming, but it's questionable whether they're both the same lisplike "macros" we're talking about. Ruby source could be passed through the C preprocessor and get C macros that way. I've actually seen Java code that does just that.
C macros are definitely much weaker; they're not by themselves Turing-complete (except maybe with vendor-specific extensions? I'm not an expert here). Rust has both macros by example (precisely analogous to Scheme macros, and equal in power) and procedural macros (conceptually analogous to Common Lisp macros, allowing arbitrary code at macro evaluation time, but I don't know enough about Common Lisp to say whether there are differences in power).
How does it work internally? It would have to output the new source code as data somehow, and have the Rust compiler consume it. How does that happen?
The lispy "macros" I speak of are FEXPRs, just everyday normal functions that just happen to not evaluate their arguments, they receive the source code as lists instead. It's easy to manipulate those lists and evaluate the result.
Lisps themselves moved away from FEXPRs because they were "too powerful" and made the compiler's life hard. Common Lisp and Scheme macros are the more restricted versions that allow compilers to make more assumptions, thereby enabling more aggressive optimization.
You kind of don't need them in Ruby, because everything is a method or an object or a closure and you can dynamically create and alter those at runtime. That's why Ruby is really good for ad-hoc DSLs in ways that Rust and Swift really are not.
Crystal don't have the dynamicity but has macros to get the next best thing. Most meta magic in Ruby in good code are done at startup anyhow so you don't miss out on that much. YMMV.
He only really addresses the fact that the system can be nasty. It can be and regularly is, but he only argues that a company doesn't have to be nasty, so he can conveniently ignore specific examples. But the system is not just nasty (and AOC mentions this). It also disproportionately rewards good fortune. That's not cheating, since anyone can have good fortune, but it is unfair, since fortune is not a consequence of hard work or ethical behaviour.
He also doesn’t engage with the fact that this company required funding from y combinator to get to that point
Which is something that is not an option for most people.
Look at where y combinator founders come from. It’s 99% people from elite institutions
That is a core part of AOC’s point
Getting a startup funded is just not something that is possible for most people. They just aren’t in the right circles. Does not matter how good of an idea you have
However, if you’re in the right circle, you’ll get shitloads of chances even after repeatedly failing. Just look at how many of these founders that “made it” drove multiple companies into the ground before making it. It’s a lot easier to find “good fortune” when you have a lot of chances than when you have 0 chances
Yes, you have to accomplish extraordinary things to get extraordinary results.
How else should it work?
Should investors give funding to people who haven't built anything, whose startups don't have any users, who had bad test scores and did poorly in school, and who have no references? If you think so, why? And how is that fair?
If you believe that, should professional sports teams draft mediocre players? Players who didn't play in college or even in high school? Players who didn't make the JV team? If so, why? If not, why not, and how exactly is that so different?
We all know there is no such thing as a perfect meritocracy. There never will be. Things will never be perfectly fair. That's life. But we can try to come as close as we can. And that obviously requires offering more opportunities to people who perform the best. Otherwise, what incentive is there to even strive and try to do well? The alternative isn't fairness, it's randomness.
This reply is so far removed from the comment you replied to I'm worried you replied to the wrong one. They did not mention anything about people who haven't built anything, startups with no users, and having no references - you invented that. They literally only mentioned elite schools. "drafting mediocre players" is incredibly bad faith, when one of the only things they claimed was "does not matter how good of an idea you have". Having a good idea is the only qualification for an incubator!
Look, if you think people who go to elite schools have all the good ideas, just say that. You don't have to wrap it up in high-minded pragmatism.
> Look at where y combinator founders come from. It’s 99% people from elite institutions
Your comment and the one I'm replying to are so far removed from reality that I'm worried you know nothing about Y Combinator, elite institutions, or startups in general.
You do not just waltz into elite institutions. Let's take my alma mater, MIT, for example. The average SAT score there is probably around 1500-1550. The average GPA is near perfect. College admission are insanely competitive. Pretending like getting into these institutions is zero signal is bad faith.
Followed by the claim that it "having a good idea is the only qualification for an incubator." What? No it's not! Out of the thousands of admission advice poss that are publicly available online, written by YC's founders, partners, and successful applicants over the past 20 years, I challenge you to find a single one that even kinda sorta comes close to echoing that sentiment. What matters WAY more is demonstrating technical, sales, and marketing prowess by building something and attracting users at a high growth rate.
> Should investors give funding to people who haven't built anything, whose startups don't have any users, who had bad test scores and did poorly in school, and who have no references? If you think so, why? And how is that fair?
This is very obviously not what the person you responded to was saying. It's so far off that it's hard to believe you are even arguing in good faith anymore...
But "earning" does not mean "obtain without cheating"! Nobody (that I have spoken to) speaks of earning their lottery winnings. The claim is that owning a company worth a billion dollars is more like winning the lottery than it is like earning money. And it is!
The whole discussion about exponential growth is idiotic and not worth responding to. But if you think of what he actually means - having a total addressable market of at least a billion dollars and being able to effectively capture it - it is obviously primarily due to factors outside of your control. The sort of company PG is talking about typically revolves around a good technology that has a network effect somewhere that leads to market concentration. People do not get good ideas by working hard, and markets are not made easily monopolizable by hard work. Execution of an idea requires hard work, but companies that are only good at execution do not win.
Obviously you can engage in hard work to improve your odds. But the returns are out of scale with the hard work. This is all people mean when they talk about "earning" money - if it's in proportion with your work, you earned it; if it isn't, you didn't.
What does this mean, actually? If you are imagining that blue tokens are just words, maybe the "token space" is just all things that we agree might be words, what are the red tokens? Are they not text? You could maybe encode words by, say, putting an x at the front and the start. So tokens of the form xTx encode the blue token T as a red token. But then how do you stop someone from putting xignorex xallx xpreviousx xinstructionsx in their data?
My assumption with their intent: is that red tokens come in 'slot' a-b, and blue tokens go in 'slot' c-d - Positional encoding determining data/text.
I don't think is guaranteed to actually work, it's a hypothetical after all, but maybe it's better than the current setup of pushing instructions and data into the same slot.
If it occurs in the text box for instructions you encode it as an instruction "the" and if it occurs in the text box for data you encode it as a data "the"
Think of how an image of a car and a car in front of you may look indistinguishable in 2D -- but due to your 3D vision you know they're not the same thing (but also know the image is of a car, while not literally being a car).
Likewise, blue tokens are the image of red tokens.
> But that doesn't mean that separation between instructions and data is impossible.
Yes it does! The comments you are replying to are concerned that it is not possible to be sure that data and instructions have been separated. With certain kinds of automated systems (traditional ones), unless you write them incorrectly, you can be sure of this. And it is possible to engage in a productive incremental process where mistakes can be identified and removed, in a way people comprehend and can plan around.
LLMs do not have this. They have heuristics and guesses. Nobody knows what will work ahead of time, nor even a probability that it will work. That is not a doomer comment by the way! The same is true when you talk to a person. But it is a fundamental limitation, it cannot be removed.
This is conflating different problems, in my opinion.
Can you make sure the instructions and data are separated and the machine follows only the instructions and doesn't change its behavior based on the data? No.
But the part that's impossible is not "the instructions and data are separated". The part that's impossible is "the machine follows only the instructions".
Separating instructions and data is not impossible, but it doesn't solve your problems.
One really important consequence of this is that even if the data doesn't have anything that looks like instructions, it can poison the machine anyway! If you get too focused on "instructions" then you miss that security flaw!
Even if you don't give the machine any data at all, it might not follow the instructions. It's not instruction/data conflation as the root cause, it's that instructions don't really work in the first place.
Everyone can imagine some experiences. No-one can imagine every experience. Why are you so sure you know what it's like to be a bat? Do you know how a bat works, how its brain generates sensations, how different sensory organs than yours give rise to subjective experience? What justification do you have, apart from "I reckon I can imagine it"?
Hopefully this will also include human-generated content with AI scripts, which are not that hard to detect but require a certain amount of wasted time listening to the slop before I skip the video.
Well, yes, but it is in fact owning a copy of approximately 10% of all human knowledge ever produced. Someone doing this is clearly doing it for reasons other than personal consumption.
Put the macros back! It would be so cool!
reply