There's going to be a spectrum here, but for some of these new pilot projects like vinext where the claim is primary AI authorship I would expect the resulting works not to be copyrighted.
What does that mean? I don't know. They are claiming copyright over vinext and licensing it under MIT, a copyright-based license. So the license and the copyright both get swept away in the flood there and what's left is a formerly-copyrighted software duplicated as a work [in the public domain?] [that nobody can legally use?] [that the author can legally use but not legally license?]
Choose your own adventure, now with copyright law!
Yeah sorry, I can claim there's none, you can't stop me.
I could claim there's even less creativity than lip syncing, and I will.
And if there was any creativity, the use it is being put to is to do violence to artists. If you think you deserve someone else's work as your own, you better be prepared for the fact that you won't even really understand who you're ripping off, but someone else sure as shit will and they're going to be pissed as hell.
At least Milli themselves a) knew what they were doing, b) paid the real singers I presume and c) presented real art created by real people.
But still everyone was mad at the lie of it, at being asked to venerate an imposter. And being asked to believe that in the future "impostor" will be the most venerable role. No. Just no.
Sure sounds like congress renamed it. Those damn masses, exercising democratic power.
Trump will put a stop to that!
Loyalty to the constitution third, loyalty to the party second, loyalty to the president first. That's the order of things in a fascist society and Trump has made very exceptionally clear that he thinks that should be the way of it in the US...
That's what worries me so much about the development that OpenAI is stepping in. OpenAI's claim is that they have the same principles as Anthropic, but that claim is easy because it's free now because the govt wants to sell the "old bad, new good" story.
Surely OpenAI cannot but notice that those values, held firmly, make you an enemy of the state?
My reading is that OpenAI is paying lip service. Altman is basically saying "OF COURSE we don't want to spy on Americans or murderdrone randos, but OF COURSE the government would never do that, they just told me so (except for the fact that they just cut ties with Anthropic because Anthropic wouldn't let them do that)"
Its much simpler than that. OpenAI is losing significant market share and this is a Hail Mary that the government will forcr troves of companies to leavr Anthropic
I'm fairly sure it's not Promises that are actually the heavy part but the `await` keyword as used in the `for await` loop. That's because await tries to preserve the call stack for debugging, making it a relatively high-level expensive construct from a perf perspective where a promise is a relatively low-level cheap one.
So if you're going to flatten everything into one stream then you can't have a for loop implementation that defensively awaits on every step, or else it'll be slooooooooow. That's my proposal for the change to the language is a syntax like
for await? (value of stream) {
}
which would only do the expensive high-level await when the underlying protocol forced it to by returning a promise-valued step.
Yeah I think people took away "It's better to be uniform" since they were trying to block out the memory of much-feared Zalgo, but if you read the article carefully it says in big letters "Avoid Synthetic Deferrals" then goes on to advocate for patterns exactly like MaybeAsync to be used "if the result is usually available right now, and performance matters a lot".
I was so sick of being slapped around by LJHarb who claimed to me again and again that TC39 was honoring the Zalgo post (by slapping synthetic deferrals on everything) that I actually got Isaacs to join the forum and set him straight: https://es.discourse.group/t/for-await-of/2452/5
That's an amazing thread, thanks for posting it! I've wanted `for await?()` for exactly these situations.
I feel like my deep dives into iterator performance are somewhat wasted because I might have made my project faster, but it's borderline dark magic and doesn't scale to the rest of the ecosystem because the language is broken.
You know now that I look at it I do think I need to change this code to defend better against multiple eager calls to `next()` when one of them returns a promise. With async generators there's a queue built in but since I'm using sync generators I need to build that defense myself before this solution is sound in the face of next();next(). That shouldn't be too hard though.
type Stream<T> = {
next(): { done, value: T } | Promise<{ done, value: T }>
}
Obviously I'm gonna be biased, but I'm pretty sure my version is also objectively superior:
- I can easily make mine from theirs
- In theirs the conceptual "stream" is defined by an iterator of iterators, meaning you need a for loop of for loops to step through it. In mine it's just one iterator and it can be consumed with one for loop.
- I'm not limited to having only streams of integers, they are
- My way, if I define a sync transform over a sync input, the whole iteration can be sync making it possible to get and use the result in sync functions. This is huge as otherwise you have to write all the code twice: once with sync iterator and for loops and once with async iterators and for await loops.
- The problem with thrashing Promises when splitting input up into words goes away. With async iterators, creating two words means creating two promises. With stream iterators if you have the data available there's no need for promises at all, you just yield it.
- Stream iterators can help you manage concurrency, which is a huge thing that async iterators cannot do. Async iterators can't do this because if they see a promise they will always wait for it. That's the same as saying "if there is any concurrency, it will always be eliminated."
> Obviously I'm gonna be biased, but I'm pretty sure my version is also objectively superior:
> - I can easily make mine from theirs
That... doesn't make it superior? On the contrary, theirs can't be easily made out of yours, except by either returning trivial 1-byte chunks, or by arbitrary buffering. So their proposal is a superior primitive.
On the whole, I/O-oriented iterators probably should return chunks of T, otherwise you get buffer bloat for free. The readv/writev were introduced for a reason, you know.
This lines up with my thinking. The proposal should give us a building block in the form of the primitive. I would expect the grandparent comment’s API to be provided in a library built on top of a language level primitive.
How would you then deal with a stream of UTF8 code points? They won't fit in a UInt8Array. There will be too many for async iterators to perform well: you'll hit the promise thrashing issues discussed in the blog post
No, you'll just need to (potentially) keep the last 1-2 bytes of the previous chunk after each iteration. Come on, restartable UTF-8 APIs has been around for more than 30 years.
But those code points were just inputs to another stream transformation that turns a stream of code points into a stream of graphemes. Rapidly your advice turns into "just do everything in one giant transformation" and that loses the benefits of streams, which are meant to be highly composable to create efficient, multi-step transformation pipelines.
What's stopping you from implementing a stream transformation that reads the raw stream like a parser, outputting a grapheme or whatever unit you want only when it knows it's done reading it from the input?
No, it doesn't turn into this. Those two bytes of leftovers plus a flag are kept inside the stream generator that transforms bytes into code points, every time you pull it those two bytes are used as an initial accumulator in the fold that takes the chunk of bytes and yield chunk of code points and the updated accumulator. You don't need to inline it all into one giant transform.
Come on, it's how (mature libraries of) parser combinators work. The only slightly tricky part here is detecting leftover data in the pipeline.
> If you want to stream arbitrary JavaScript values, use async iterables directly
OK, so we have to do this because code points are numbers larger than 8 bits, so they're arbitrary JS values and we have to use async iterables directly. This is where the amount of per-item overhead in an async iterable starts to strangle you because most of the actual work being done at that point is tearing down the call stack between each step of each iterator and then rebuilding it again so that the debugger has some kind of stack traces (if you're using for await of loops to consume the iterables that is).
As an abstraction I would say it does make mine superior that it captures everything theirs can and more that theirs can't.
Plus theirs involves the very concrete definition of an array, which might have 100 prototype methods in JS, each part of their API surface. I have one function in my API surface.
I think the more generic stream concept is interesting, but their proposal is based on different underlying assumptions.
From what it looks like, they want their streams to be compatible with AsyncIterator so it'd fit into existing ecosystem of iterators.
And I believe the Uint8Array is there for matching OS streams as they tend to move batches of bytes without having knowledge about the data inside. It's probably not intended as an entirely new concept of a stream, but something that C/C++ or other language that can provide functionality for JS, can do underneath.
For example my personal pet project of a graph database written in C has observers/observables that are similar to the AsyncIterator streams (except one observable can be listened to by more than one observer) moving about batches of Uint8Array (or rather uint8_t* buffer with capacity/count), because it's one of the fastest and easiest thing to do in C.
It'd be a lot more work to use anything other than uint8_t* batches for streaming data. What I mean by that, is that any other protocol that is aware of the type information would be built on top of the streams, rather than being part of the stream protocol itself for this reason.
Yeah it makes sense to me that the actual network socket is going to move data around in buffers. I'm just offering an abstraction over that so that you can write code that is wholly agnostic to how data is stored.
And yes, because it's a new abstraction the compat story is interesting. We can easily wrap any source so we'll have loads of working sources. The fight will be getting official data sinks that support a new kind of stream
I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case.
If you go back a few versions, that number goes up to around 105x. I don’t recall now if I tested back to 14. There was an optimization to async handling in 16 that I recall breaking a few tests that depended on nextTick() behavior that stopped happening, such that the setup and execution steps started firing in the wrong order, due to a mock returning a number instead of a Promise.
that sounds way off. there is a big perf hit to async, but it appears to be roughly 100 nanoseconds overhead per call. when benchmarking you have to ensure your function is not going to be optimized away if it doesn't do anything or inputs/outputs never change.
> I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case.
Any await runs the logic that attempts to release the main message pump to check for other tasks or incoming IO events. And it looks like that takes around 90 instructions to loop back around to running the next line of the code, when the process is running nothing else.
If you’re doing real work, 90 instructions ain’t much but it’s not free either. If you’ve got an async accumulator (eg, otel, Prometheus) that could be a cost you care about.
How did you come up with 90? Can you shed any might on the difference between the cost of promise resolution and the cost of await? Is there any cost component with how deep in the call stack you are when an await happens?
Essentially for loop of 10k iterations comparing `fn()` versus `await fn()` fed into a microbenchmark tool, with some fiddling to detect if elimination was happening or ordering was changing things.
I was bumping into PRs trying to eliminate awaits in long loops and thinking surely the overhead can’t be so high to warrant doing this, especially after node ~16. I was wrong.
I assume the reason is that `await` de-schedules the current microtask. In fact, even if you immediately return from an `await`, the de-scheduling can introduce behavior that otherwise would be absent without `await`. For this reason, code optimizers (like the Google Closure Compiler) treat `await` as a side-effect and do not optimize it out.
JS engines actually are optimized to make that usage pattern fast.
Small, short-lived objects with known key ordering (monomorphism) are not a major cost in JS because the GC design is generational. The smallest, youngest generation of objects can be quickly collected with an incremental GC because the perf assumption is that most of the items in the youngest generation will be garbage. This allows collection to be optimized by first finding the live objects in the gen0 pool, copying them out, then throwing away the old gen0 pool memory and replacing it with a new chunk.
What happens when I send an extremely high throughput of data and the scheduler decides to pause garbage collection due to there being too many interrupts to my process sending network events? (a common way network data is handed off to an application in many linux distros)
Are there any concerns that the extra array overhead will make the application even more vulnerable to out of memory errors while it holds off on GC to process the big stream (or multiple streams)?
I am mostly curious, maybe this is not a problem for JS engines, but I have sometimes seen GC get paused on high throughput systems in GoLang, C#, and Java, which causes a lot of headaches.
Yeah I don't think that's generally a problem for JS engines because of the incremental garbage collector.
If you make all your memory usage patterns possible for the incremental collector to collect, you won't experience noticeable hangups because the incremental collector doesn't stop the world. This was already pretty important for JS since full collections would (do) show up as hiccups in the responsiveness of the UI.
Interesting, thanks for the info, I'll do some reading on what you're saying. I agree, you're right about JS having issues with hiccups in the UI due to scheduling on a single process thread.
Makes a lot of sense, cool that the garbage collector can run independently of the call stack and function scheduler.
OP doesn’t know what he’s talking about. Creating an object per byte is insane to do if you care about performance. It’ll be fine if you do 1000 objects once or this isn’t particularly performance sensitive. That’s fine. But the GC running concurrently doesn’t change anything about that, not to mention that he’s wrong and the scavenger phase for the young generation (which is typically where you find byte arrays being processed like this) is stop the world. Certain phases of the old generation collection are concurrent but notably finalization (deleting all the objects) is also stop the world as is compaction (rearranging where the objects live).
This whole approach is going to be orders of magnitude of overhead and the GC can’t do anything because you’d still be allocating the object, setting it up, etc. Your only hope would be the JIT seeing through this kind of insanity and rewriting to elide those objects but that’s not something I’m aware AOT optimizer can do let alone a JIT engine that needs to balance generating code over fully optimal behavior.
Don’t take my word for it - write a simple benchmark to illustrate the problem. You can also look throughout the comment thread that OP is just completely combative with people who clearly know something and point out problems with his reasoning.
Even if you stop the world while you sweep the infant generation, the whole point of the infant generation is that it's tiny. Most of the memory in use is going to be in the other generations and isn't going to be swept at all: the churn will be limited to the infant generation. That's why in real usage the GC overhead is I would say around 15% (and why the collections are spaced regularly and quick enough to not be noticeable).
I've been long on JS but never heard things like this, could you please prove it by any means or at least give a valid proof to the _around 15%_ statement?
Also by saying _quick enough to not be noticeable_, what's the situation you are referring too? I thought the GC overhead will stack until it eventually affects the UI responsiveness when handling continues IO or rendering loads, as recently I have done some perf stuff for such cases and optimizing count of objects did make things better and the console definitely showed some GC improvements, you make me nerve to go back and check again.
Yeah I mean don't take my word, play around with it! Here's a simple JSFiddle that makes an iterator of 10,000,000 items, each with a step object that cannot be optimized except through efficient minor GC. Try using your browser's profiler to look at the costs of running it! My profiler says 40% of the time is spent inside `next()` and only 1% of the time is spent on minor GCs. (I used the Firefox profiler. Chrome was being weird and not showing me any data from inside the fiddle iframe).
The allocation of each object still has overhead though, even if they all live side-by-side. You get memory overhead for each value. A Uint8Array is tailor-made for an array of bytes and there’s a constant overhead. Plus the garbage collector doesn’t even have to peer inside a Uint8Array instance.
The engine can optimize all those allocations out of existence so they never happen at all, so it's not a problem we'll be stuck with forever, just a temporary inconvenience.
If a generator is yielding values it doesn't expose step objects to its inner code. If a `for of` loop is consuming yielded values from that generator, step objects are not exposed directly to the looping code either.
So now when you have a `for of` loop consuming a generator you have step objects which only the engine ever can see, and so the engine is free to optimize the allocations away.
The simplest way the engine could do it is just to reuse the same step object over and over again, mutating step.value between each invocation of next().
It's not blazingly fast, no, but it's not as much overhead as people think either when they're imagining what it would cost to do the same thing with malloc. TC39 knew all this when they picked { step, done } as the API for iteration and they still picked it, so I'm not really introducing new risk but rather trusting that they knew what they were doing when they designed string iterators.
At the moment the consensus seems to be that these language features haven't been worth investing much in optimizing because they aren't widely used in perf-critical pathways. So there's a chicken and egg problem, but one that gives me some hope that these APIs will actually get faster as their usage becomes more common and important, which it should if we adopt one of these proposed solutions to the current DevX problems
That's just how string iterators work in Javascript: one object for every byte. For now it's fast enough: https://v8.dev/blog/trash-talk. I'd put the GC overhead at around 10-15%, even with 20+ objects per byte when you add up all the stages in a real text processing pipeline. It's that cheap because the objects all have short lifespans and so they spend all their lives in the "tiny short-lived objects" memory pool which is super easy to incrementally GC.
In the future it should be entirely possible for the engines to optimize even more aggressively too: they should be to skip making the object if the producer of values is a generator function and the consumer is a for loop.
I agree with your post, but in practice, couldn't you get back that efficiency by setting T = UInt8Array? That is, write your stream to send / receive arrays.
My reference point is from a noob experience with Golang - where I was losing a bunch of efficiency to channel overhead from sending millions of small items. Sending batches of ~1000 instead cut that down to a negligible amount. It is a little less ergonomic to work with (adding a nesting level to your loop).
> Adding types on top of that isn't a protocol concern but an application-level one.
I agree with this.
I have had to handle raw byte streams at lower levels for a lot of use-cases (usually optimization, or when developing libs for special purposes).
It is quite helpful to have the choice of how I handle the raw chunks of data that get queued up and out of the network layer to my application.
Maybe this is because I do everything from C++ to Javascript, but I feel like the abstractions of cleanly getting a stream of byte arrays is already so many steps away from actual network packet retrieval, serializing, and parsing that I am a bit baffled folks want to abstract this concern away even more than we already do.
I get it, we all have our focuses (and they're ever growing in Software these days), but maybe it's okay to still see some of the bits and bytes in our systems?
My concern isn't with how you write your network layer. Use buffers in there, of course.
But what if you just want to do a simple decoding transform to get a stream of Unicode code points from a steam of bytes? If your definition of a stream is that it has UInt8 values, that simply isn't possible. And there's still gonna be waaay too many code points to fall back to an async iterator of code points.
I think we're having a completely different conversation now. The parent comment I originally replied has been edited so much that I think the context of what I was referring to is now gone.
Also, I wasn't talking about building network layers, I was explicitly referring to things that use a network layer... That is, an application receiving streams of enumerable network data.
I also agree with what you're saying, we don't want UInt8, we want bits and bytes.
I'm really confused as to why the parent comment was edited so heavily. Oh well, that's social media for you.
Not the person originally replying, but as someone who avoids JS I have to ask whether the abstraction you provide may have additional baggage as far as framing/etc.
Ironically, naively, I'd expect something more like a callback where you would specify how your input gets written to a buffer, but again im definitely losing a lot of nuance from not doing JS in a long while.
Other angles of critique & consideration already covered well by sibling commenters. One extra consideration (unrelated to streams, more general) is the API design & dev UX/DX:
type Stream<T> = {
next(): { done, value: T } | Promise<{ done, value: T }>
}
the above can effectively be discussed as a combination of the following:
type Stream<T> = {
next(): { done, value: T }
}
type Stream<T> = {
next(): Promise<{ done, value: T }>
}
You've covered the justifications for the 2nd signature, but it's a messy API. Specifically:
> My way, if I define a sync transform over a sync input, the whole iteration can be sync making it possible to get and use the result in sync functions. This is huge as otherwise you have to write all the code twice: once with sync iterator and for loops and once with async iterators and for await loops.
Writing all the code twice is cleaner in every implementation scenario I can envisage. It's very rare I want generalised flexibility on an API call - that leads to a lot of confusion & ambiguity when reading/reviewing code, & also when adding to/editing code. Any repetitiveness in handling both use-cases (separately) can easily be handled with well thought-out composition.
How is it cleaner? I used to actually do that. I wrote everything twice. I even built fancy tools to help me write everything twice.
But the bigger problem here is that sync and async aren't enough. You almost need to write everything three times: sync, async, and async-batched. And that async-batched code is gonna be gnarly and different from the other two copies and writing it in the first place and keeping it in sync is gonna give you headaches.
To see how it played out for me take a look at the difference between:
In the language I've been working on for a couple months, Eidos, streams are achieved through iterators as well. It's dead simple. And lazy for loops are iterators, and there is piping syntax. This means you can do this (REPL code):
>> fn double(iter: $iterator<i32>) {
return *for x in iter { $yield( x * 2 )}
}
>> fn add_ten(iter: $iterator<i32>) {
return *for x in iter { $yield( x + 10 )}
}
>> fn print_all(iter: $iterator<i32>) {
for x in iter { $print( x )}
}
>> const source = *for x in [1, 2, 3] { $yield( x )}
>> source |> double |> add_ten |> print_all
12
14
16
You get backpressure for free, and the compiler can make intelligent decisions, such as automatic inlining, unrolling, kernel fusing, etc. depending on the type of iterators you're working with.
I think the context that some other responders are missing is that in some functional languages, like Elixir, streams and iterators are used idiomatically to do staged transforms of data without necessitating accumulation at each step.
They are those languages versions of goroutines, and JavaScript doesn’t have one. Generators sort of, but people don’t use them much, and they don’t compose them with each other.
So if we are going to fix Streams, an implementation that is tuned only for IO-bound workflows at the expense of transform workflows would be a lost opportunity.
There's one more interesting consequence: you rid yourself of the feedback problem.
To see the problem let's create a stream with feedback. Lets say we have an assembly line that produces muffins from ingredients, and the recipe says that every third muffin we produce must be mushed up and used as an ingredient for further muffins. This works OK until someone adds a final stage to the assembly line, which puts muffins in boxes of 12. Now the line gets completely stuck! It can't get a muffin to use on the start of the line because it hasn't made a full box of muffins yet, and it can't make a full box of muffins because it's starved for ingredients after 3.
If we're mandated to clump the items together we're implicitly assuming that there's no feedback, yet there's also no reason that feedback shouldn't be a first-class ability of streams.
What does that mean? I don't know. They are claiming copyright over vinext and licensing it under MIT, a copyright-based license. So the license and the copyright both get swept away in the flood there and what's left is a formerly-copyrighted software duplicated as a work [in the public domain?] [that nobody can legally use?] [that the author can legally use but not legally license?]
Choose your own adventure, now with copyright law!
reply