-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Why just on iterators and not on arrays? #1
Comments
See the README:
Arrays are iterable, so they would "just work" if we answer "yes" to this question. Also, even if we answer "no", it's pretty easy to turn arrays into iterators with |
Well, you'd get an iterator out rather than an array, presumably? So you'd need to |
What I mean is, why not have these methods also on Array/Array.prototype? |
We could do that, but I think the |
Most of my need for zip starts with two arrays and ends with one array, and it seems useful to me to avoid the iterator protocol entirely when it’s not needed. |
Having it on Array prototype would probably allow various optimizations in the JS engine, which might be impossible with the generic iterator version. |
I wouldn't believe such claims unless they came directly from the implementors. The engines already keep taint bits on things like Symbol.iterator on built-ins and then short-circuit these kinds of operations to optimise when passed arrays or other built-ins. I don't see why they wouldn't do the same here. |
I thought it's impossible to know beforehand that the developer wants to produce an array in the end so the engine would have to speculate, which is wasteful. |
Closing this as we've taken the |
@michaelficarra i don't think that addresses the ergonomics issue nor precludes also adding the method on Arrays. |
@ljharb You're right, this proposal does not preclude another proposal that would add a similar static method on |
Why not? This issue shouldn’t be closed until that original question is answered. |
I agree that it should be added to |
Mozilla has already said they'll reject anything adding to |
@jridgewell that was not my interpretation of what they said; it was that new methods on Array.prototype would require a stronger motivation.
|
Can you give an example of a time when you'd want an array as the result? In my experience, |
I don't have a concrete example off hand - i'll certainly try to find one - but i definitely use "zip" at times as not a step in a chain (where "a chain" means "within the same function/scope") - and i want to pass around arrays, not iterators, basically 100% of the time. I only use iterators as part of an inline transformation. |
I don't know how much there is to lose by adding IMO Of course, |
This is a big "if". I agree with @bakkot that the zip result is almost always used as an intermediate result. If that's true, we should not make it as easy to use the version that realises an array. It's easy enough to do |
Looks like a very heavy-handed way of helping the programmers. According to this logic there should be no Array.prototype.map, filter, and other established methods because they are often chained and produce an unnecessary array. And although I sometimes avoid using these methods for this exact reason, it doesn't mean I agree that this functionality shouldn't be present in the language as it's useful in case where performance/memory/GC is not a concern. |
I don't know if that's what @michaelficarra is getting at. It sort of feels like the opposite, i.e. optimization isn't a concern here and we are talking much more about expression rather than performance (I interpret "engines can certainly optimise this case" as saying " I don't understand what "better that it calls out the eagerness" means. Are you indicating that being forced to use If that's the case, then I kind of agree with @tophf that it's heavy-handed. I don't think it's a good idea to consider iterator operations as the absolute replacement for array operations - I don't want to accuse you of that, mind! but if it's in line with your thinking, then it feels like the wrong angle to me. We have array methods and yes, those are a legacy inclusion because they've been around forever, and yes, there are a ton of cases where iterators are more appropriate and better. But array methods still exist, and I feel as though introducing a behavior for iterators without also bringing along the direct analogue in arrays would be a missed opportunity. I feel the onus should be on the developer to learn and identify whether arrays or iterators are more appropriate for their use case, including the skill to effectively identify not just "if iterators are better than arrays", but in what specific instances their code would benefit from using each. I don't think the spec should be arbitrarily lending preference to one choice or the other if there is not an extremely clear reason that zipping is fundamentally an operation that does not make sense for arrays. |
The criteria for adding something is not "there is no identifiable reason that this operation does not make sense here". It's "this operation makes sense and is a common enough need to warrant adding something to the language, rather than expecting programmers to use the next best option". And I'm just not convinced that it is actually particularly common to need |
There's historical [1]: iteration protocol is already seamlessly integrated in array syntax like |
(edit: Just to clarify, all my comments are made in the context of the Stage 3 Iterator Helpers proposal, which I was taking as common knowledge since this is also an iterator helper, just a new kind of operation rather than an equivalent to an existing array helper.) I agree with @tophf's point above but I think it would help to have a lot more examples shared anyway. We should challenge and try to justify the historical parity going forward, rather than take it as a (functionally weaker) given, too! Real-world use cases:
Use-cases that I don't think are that strong:
I admit that in our own code the vast majority of the uses of our I'd love to have these use cases expanded upon. I strongly disagree with the premise that we need to justify retaining parity with concrete, real cases... (Wasn't much the point of iterator helpers to improve parity? Is there not an inherent value in making two API surfaces overlap with plainly analogous behavior in both? Don't we want to make room for programmer expression, rather than say, no, this is the way this works, iterators are arrays 2.0 and we won't help you use arrays anymore?) ...But I still think we should try to make that case. Losing sight of why parity is a good thing only hurts the case for parity overall, and keeps the conversation on new iterator features in general from having nearly as much perspective as is deserved. |
Feedback from committee was to not include Array-specific handling in this proposal. Closing. |
@michaelficarra um, that's not my recollection of the outcome? There's nothing in the notes that states that as a foregone conclusion, and I consider this something to resolve within stage 2, that will block stage 3. |
This comment was marked as resolved.
This comment was marked as resolved.
@devsnek can you elaborate on your comment? did you mean, you think it should be a separate proposal, or that it shouldn't happen at all? |
@ljharb To be clear, I don't think this was like a "1 vs 0" thing. Since the default state was to not do anything, I took the complete lack of expressed support as a strong negative signal. |
I was pretty clear that I'd be doing a followup if it wasn't part of this proposal, and that the cross-cutting concerns would mean that their fates were linked anyways, would would effectively mean that this proposal couldn't advance to stage 2.7 until that one hit stage 2. At that point I'm not sure why a separate proposal adds value. |
@ljharb Ideally I'd prefer that we aren't duplicating iterator methods on arrays just for convenience. If there's an issue expressiveness or performance, I think that's worth discussing, but I'm not sure either point has been raised yet. |
Gotcha - why not? Most all of the iteration helpers are for convenience. (it's certainly also about expressiveness and performance, and a number of other things, but i think "convenience" is pretty compelling on its own) |
I agree with @ljharb's argument above and especially wtih @tophf in #1 (comment). Parity is a big deal for iterator helpers. In my mind, these proposals bring the same functionality to iterators as you're used to working with in arrays. If iterators are not intended to canonically serve as the replacement for arrays, then we shouldn't give preference to adding a useful feature for iterators but not for arrays — namely because that goes against the parity that we're establishing through the rest of the iterator helpers. Here are the helpers in Iterator Helpers:
Here are the other methods on Array.prototype that aren't paired in Iterator Helpers:
I put this table together to substantiate or counter my argument — to actually assess the overall parity. And yeah, basically everything here fairly obviously doesn't work as an iterator method, so isn't part of Iterator Helpers. I can only really make cases for Going counter to that parity because we don't have effective circumstantial justification / precedent for |
@towerofnix The intention of the iterator helpers proposal was not to "bring parity with array methods". It was to take a common interface shared among a great deal of the ecosystem and make it useful by default for js developers. Whether or not array has a certain method, while certainly not irrelevant to the discussion, should not be unto itself a reason to include or deny an iterator method. And this argument applies the other way too, just because iterators have a certain method doesn't mean that arrays (or any other collection type) must include it. @ljharb I just don't think its particularly burdensome to jump between arrays and iterators. Maybe rust has desensitized me from typing |
The danger of not maintaining the historical parity is that it will confuse developers as there's no logical substantiation for why some methods are missing in Array/Iterator. After developers needlessly suffer for a while a new proposal would appear to bring these new methods to arrays, similarly to the ongoing Iterator Helpers proposal. |
TBF there isn't really "historical" parity, because Iterator Helpers and this proposal are both new additions to the language. I think parity is a good idea in general, and I feel that even if it wasn't part of the designed intent of Iterator Helpers there does appear to be a lot of parity where it makes sense. But those are also just my opinion. If someone sees I do think developers are going to have personal experiences that lead them to expect parity between obviously-analogous array and iterator functions, and IMO I'd like to say it should always matter as something that's fundamental to how people learn programming languages (and interact with them in general), but that's a somewhat high-and-mighty perspective if I haven't done research or at least got anecdotal experience beyond the way I've learned programming languages. |
@devsnek i think it's totally fine if users want to only use the iterator interface. I definitely don't think that preference should have any bearing on the separate existence of an array interface, for those that prefer that. In general, "it's not personally interesting to me" isn't a persuasive reason not to progress on any functionality. |
As a general principle, I also prefer to work with arrays where cache locality matters. But that argues that the use cases ought to be evaluated case-by-case. I am not sure zipping in particular is an operation I care about from a cache locality perspective. Should I? |
I'm not sure what "cache locality" means here, and I don't think I have that use case - i definitely zip arrays together on a small number of projects, frequently. |
I'm happy with either direction and am leaning towards omitting given the ease of calling By "cache locality" I mean if I care about the performance of accessing adjacent elements (both adjacent in their index and in time of access), I'll pay the up-front memory of the array. So @michaelficarra I was really asking you if I should care about zipping in that performance context. I still see mostly abstract discussion of use cases. @bakkot has a comment up-thread about zipping mostly being an intermediate step, which would make me think the per-item processing takes enough time that the performance of accessing adjacent, zipped elements isn't usually important. |
@syg In that kind of performance context, you're probably going to try to avoid creating any kind of intermediate structure. Zip is useful when describing your program as a series of data transforms. In pure FP languages, those transforms get fused and the intermediate structures are never actually realised. But this is JavaScript, so if you want to achieve performance characteristics similar to what you'd get if you were manually jointly iterating (assuming you are doing further processing after your zip), you'd have to actually do just that.
This is probably usually true, which is another reason that Array zipping is not motivated by performance IMO. However, while you seem to want to use performance to justify Array zipping to yourself, I don't believe @ljharb was making a (solely) performance-based argument for it. From above,
If you find that unconvincing, we can omit it for now and pursue it as a follow-up. My goal here is to not put the Iterator-based zipping portion of this proposal at risk of not advancing. |
I don't find the convenience argument convincing by itself because I think |
It's definitely not solely or even primarily performance-based; it's about the mental model. Iterator.zip(a, b).toArray()
Array.zip(a, b) Both are more or less identically convenient, but why should i have to reach for Iterator when i only want to work with arrays? |
Yeah, and providing a standard language for that mental model is exactly why Like, if you only work with arrays and are not interested in interacting with iterators, then there's no reason your own project can't export a utility for that in some convenient-to-access place: // util.js
export function arrayZip(...args) {
return Iterator.zip(...args).toArray()
} But then it's up to each program to decide what name it wants to use for "array zip", and there aren't any good choices:
Of course, each program that decides to make up a shorthand name will likely use a different one, and some just won't and will keep using It's impossible to get ahead of the curve on every possible "simple function", but because You can more or less address supporting your particular mental model using a shorthand utility function, but you certainly won't be using the same language as everyone else. Or you can hope to use the same language as people "should" use (longhand |
Hey, just caught up on the TC39 meeting minutes from April 8 and April 11—didn't see those til now, despite our more recent reply! (I guess these weren't prepared and online til a week ago, so no surprise there in the end. tc39/notes@5282761) I'm not too worried about
I agree that it's of course a good idea for developers to be attentive about this. I just don't think it makes sense for the language to be, like, forcing a developer to make that choice, just because of a difference in API surface. That doesn't seem like a very useful precedent and I think it would be an un-fun time arguing against that when an Other than that I feel totally OK with this going either way for now. I hope if |
@towerofnix I don't understand your reply to my comment. I wasn't talking about teachability, but the performance cost of zipping giant things. That is, zipping giant arrays means holding the entire zipped result in an array, which uses a lot of memory. If most of the time, the point of zipping giant lists of things is to process the pairs one at a time, you do not want an array. |
Yeah sorry, I can see that my point wasn't clear. What I meant is: That's a meaningful observation about performance, but it's something developers have to learn for themselves, so they can decide whether an iterator-based operation or an array-based operation is more suitable. Like it was discussed, for some kinds of data/operations zipping as an iterator is better, other times zipping into an array is. I could be misreading the minute here but that's what I took from your comment:
edit, also from @ljharb's reply to the above, emphasis mine:
Those trade-offs are things a dev should be aware of so they have to learn what they are, but they don't need to learn by being "forced" to consider the trade-off by the language, if that makes sense. Like yeah, totally agreed with "If most of the time, the point of zipping giant lists of things is to process the pairs one at a time, you do not want an array." But zip isn't going to be used just for giant lists of things, it will also be used for comparatively very small lists of things, and the memory overhead of having all those things exist at once is not an issue. (Could be preferable compared to CPU overhead of initializing and processing an iterator, esp. if you're just So 1) there may be a case for IMO the symmetry (for like operations, not in general) between arrays and iterators already does a good job of encouraging you to consider if iterators are better for you, i.e. if your process could be represented similarly with iterators, then JavaScript is going to make it easy for you (by giving |
Thanks for the explanation. I see where you're coming from. The general principle here is "performance footgun", where browsers especially (I represent V8/Chrome) don't want the slow thing to be too easy to reach for. The thinking goes as you've laid out: that the lack of availability would be met with some surprise and the developer would give extra thought to why there isn't symmetry. It is reasonable to disagree with the above principle. At V8 we've drawn the opposite conclusion because performance, AFAICT, is by and large deprioritized by most web developers. The MO is to ship and to ship fast. Performance matters for the tail of very mature products, certainly, but what we've observed is that performance engineering is mostly a luxury -- features, correctness, etc, usually take priority. If there's a choice in APIs, the more convenient one usually wins out. As a browser, we have a goal in making the web browsing experience, and therefore websites, performant. Taken together, that's why we prefer to not spec potential performance footguns, especially if it's a matter of convenience and not expressibility. I disagree with the "forced/strongly pushed" framing. |
Sounds like this approach may turn JS into Java with its super verbosity that's still not preventing the apps from being very slow. Anything can be a performance footgun in any language, JS included, even the best practices can be applied incorrectly. |
I think ^ what @tophf says is true in a strict sense... but I also totally understand where the reasoning comes from. I wouldn't want to make the assumption that this sort of API surface (let's say as a decision by ECMAScript, affecting a basic interface in the language, not a more specialized web API) truly has an impact on the observed performance of apps or the broad experience of devs writing those apps wrt performance. But, the reasoning is sound: reach for the easiest API, and I think it's very likely there is a general precedence for, like, making one operation easier than another, by making the latter operation composed out of two smaller parts (e.g. I think it'd be worth finding/sharing cases of that in general (for calls ECMAScript has made), but also to do with performance in specific. IMO performance being one of the weakest points of products made for the web (which feels like "basically all products", to us people who are always surrounded by the web, LOL)—the impact that ECMAScript decisions has on that should be assessed more closely, more rigorously. Best not to assume that your reasoning, despite being sound, actually has the impact you are going for, especially in a crucial area. I don't say any of this to suggest anyone is making these assumptions, just sharing my angle! I don't want to come across with an aggressive tone, "examples or it didn't happen", but I do think identifying the impact ECMAScript decisions have practically had on performance would be pretty crucial to strengthening or weakening this argument, and a worthwhile effort. I'm also OK with giving this more weight than I initially did, like, I do think aiding developers to write more performant code for the web - a context where no one writes performant code in part because it's so easy not to - is a valid concern. (The question is "how much 'in part', and how much can ECMAScript have an impact on this?") I think this is a good angle to figure out in more detail and that, uh, it may require a significant amount of effort and discussion both outside and inside committee, that would be best suited on a proposal which is maybe pivotally affected by this angle. Not treated as something that blocks progress on Oh right, and obviously you can't go back on adding |
The committee decided to move forward with this proposal without the Array variants at the June plenary. Closing. |
I have this need on Arrays just as often as on non-array iterators; any reason not to add it to both?
The text was updated successfully, but these errors were encountered: