LLMs still hallucinate when context windows become an issue.
LLMs will decide to “cheat” tests in order to get them to work from time to time. I have seen the LLM just remove functionality as a way to “fix” bugs in that code. While this might not technically be “lazy”, it results in the same issues.
Will humans do these same things? Yes, from time to time. But it is important to know that LLMs have real limitations and make mistakes, especially in reasoning and long term context.
Also, LLMs tend to agree with what the user thinks, and this can lead to poor understanding on the users part getting propagated into bad logic in the code.
Let’s be real here: unlike humans, LLMs are not capable of being lazy because they are in fact not people and do not have human emotions, thoughts, or anything near or remotely resembling anything human.
The continued anthropomorphizing of them is marking crap to make their shortcomings more acceptable. I wish we, actual real human beings, would stop talking about these computer programs as if they were people.
P.S. you’re now training the war machine. You think anthropic is different from OpenAI? Wrong.
I find it troubling that you try to shame and guilt people by those statements.
The war machine is already trained. LLM’s are already ready for mass-propaganda. LLM’s are already scraping GitHub. Even if we set a policy not to use LLM’s but only human code, we are training an LLM if a developer makes push to GitHub. Properly by posting on this forum we are training an LLM.
Either the person doesn’t believe it so they feel no guilt, or they know it’s true and feel guilt while using it, either way, when was the last time a tech this disrupting and widely adopted got adopted for autonomous killing systems (OpenAI DoW)?
If the LLMs are so good, why do we need to worry? We wont really need maintainers. We can just have Claude do it.
I do not mean that to sound as flippant as it probably does. I am sincere in asking, what is the point of worrying about dev time/sanity if the near future sees that work shifted to a bot?
Put a module freeze in place until GTK 4 conversion is complete maybe? That is a multi-year project which will allow Claude to get to a point of being able to do everything after…right??
The truth is, the major players, which are the only ones with good programming models, either are horrible unethical companies in of themselves (google, meta, xAI) or have ties to extremely bad individuals (openAI, anthropic). That leaves chinese models which are also inherently problematic, or Mistral which just isn’t good enough at the moment for serious work.
So Mica does well in pointing out these severe issues. LLMs are just too addicting and seductive. If you feel guilt because someone is pointing out facts, that’s your personal problem
No, Mica, is forcing his opinion on people by repeatedly calling people that are supporting the ‘war machine’. But I can’t see him explaining where LLM’s are being used in the ‘the war machine’. Where is he pointing about the facts?
The last time that tech was so disruptive in killing people? Well 1945… when the US dropped the atom bomb…
No, Anthropic made an opportunistic decision, which seems to have paid off.
When I was at the University teaching computers to recognize semantic roles, the ideas in the papers that me and others like me wrote were used by governmental agencies to implement algorithms to spy on people’s communications.
The same image processing algorithms (purely deterministic, pixel based stuff) that we love and cherish are used for drone and missile targeting.
Literally anything that has some technological value can and will be weaponized or used against the common good. It is naive (and possibly dishonest) to assign an ethical value to technology, Technology is neutral, it’s usages are not.
This is obviously true and any tech we use today has uses in any military around the world. Are we complicit because we use internal combustion engines that are also used in shahed drones?
What is relevant here is that we are not buying our engines from Lockheed or Raytheon, yet, the LLM services we use are indeed owned by corporations that serve the same interests, or worse, as those defense contractors.
EDIT: As I have said in other posts, I personally use copilot at work since I need to meet productivity demands and cannot really refuse it, but the moral question is clear and I don’t feel so good in doing so. So I am not trying to argue from moral superiority, for the contrary.
Not completely true in my opinion… Some technologies are almost only exclusively for military use… the technology to create highly enriched uranium is such an example. Or radar absorbing materials… can hardly think of a non-military application for that.
Yes that is the sad part. Much of the algorithm in darktable are also used in those kind of weapon system. From my perspective, the advances in ‘computer vision’ are much more troubling exactly. And those technology are essential in target selection etc.
No. The tech used to enrich uranium enough for weapons is the same tech used to enrich for energy purposes. The only real world difference is volume and speed of enrichment. The underlying tech is the same.
The killer machine argument is interesting but definitely not new.
When your parents bought you your first Texas Instruments calculator they sponsored the war machine. How? TI used the revenue to develop better chips and some of those chips were used to build even more deadlier missile guidance. The same goes for purchasing any PC. PC chip vendors use the revenue to design chips used in super-computer which are used to model and optimize nuclear weapons.
I am one of several maintainers of another popular open source project. My thoughts on use of AI agents are roughly as follows:
AI usage must be disclosed so that it can be taken into account during code review, since AI agents make mistakes that are very different in nature from the kinds of mistakes that humans make.
A contribution is only as good as the person submitting it, even when AI agents are involved. If I don’t trust a contributor to submit a good PR themselves and I can’t justify to expend the time necessary to get a PR into a polished and mergeable state, then an AI agent being involved will not fundamentally change that evaluation. With the current state of the technology, the level of competence of the human operator of an agent still dictates the quality of the design of a feature or an architecture, whether there are fundamental design flaws or bugs and whether the resulting code is maintainable.
If I trust someone to be a responsible contributor, then I can also trust them to use AI agents responsibly and not just submit AI output that they haven’t carefully checked themselves or where they weren’t deeply involved in the design. I’ll adjust the way I review contributions of this nature to account for the fact that an AI agent was involved, but I am not fundamentally against contributions involving AI agents that were instrumented by trustworthy contributors.
When an AI agent is involved in a contribution to a significant degree, my expectation for the amount of QA (e.g. in the form of tests) that a contribution must provide to be accepted increases. This is both because a higher degree of QA is needed to obtain a reasonable degree of confidence for a contribution significantly involving an AI agent, but also because AI agents reduce the amount of effort necessary for decent QA (e.g.: documenting code in detail will generally improve the quality of tests generated by an AI agent and reading a bunch of test cases for whether they test everything important correctly is generally much easier than writing loads of test cases yourself).
I much prefer contributions that utilize agents for finding and fixing bugs, for generating disposable code that can mostly just be thrown away or re-generated when it breaks or becomes unmaintainable, and for code that never ends up with users and is only utilized for development itself (tests, CI, scripts, etc.), over contributions that use them to design and implement larger or critical features. The easier it is to review and the less of a burden on maintenance it incurs, the better.
Ultimately, AI agents can increase productivity and significantly reduce the amount of time that we need to spend on some of the particularly painful bits, but we are still very much responsible for every single line of code that we commit and ship to users. This responsibility is not exclusive to the maintainers of an open source project - it extends to every single contributor, and it must be taken seriously, even when an AI agent does some of the thinking and programming for you.
For the project I work on, this means that if you are submitting a contribution that significantly involves an AI agent, you yourself must deeply understand the design and the implementation of the contribution that you are submitting.
Another honest question which seems relevant to @kofa concerns…One of the organizing principles of FOSS is the freedom to fork and do what you want with the code. Why is someone forking darktable an issue?
What worries me about anthropomorphization is making the LLM accountable for its product the same way we would a human. You can’t really deal with one in legal terms like you can a human, so expecting some fiduciary commitment ain’t gonna work. Also, if you rely on one as a contributing dev, you have to work on the realization that their “owner” can shut them down at a moment’s notice. I guess I wouldn’t take pulls from a LLM without acknowledgement from the owner that they’re on the hook.
Now, as a contributory tool to a human’s effort, I know that can be useful. Done it myself writing an OpenGL renderer for a simple CAD program I’m futzing with, it gave me explanations and code snippets I could understand and incorporate. I did just give up on one persnickety task, writing a .stl reader, I just told it I needed one and it gave me a good one. I reviewed the code, no worms or backdoors, so, yay. If I ever post this app to github I’ll disclose the LLM contribution, same as I would any other licensed code I used.
Anyhoo, I like good tools, but I don’t want to make friends with 'em…
It’s a matter of time before you will be able to serve similar models locally. The latest Qwen is already pretty good.
Regardless, let’s not be naive. All corporations are evil. They pollute, they exploit, they evade taxes, they thrive on war, they bribe and corrupt, you name it. And we all buy items or services from them. We all finance their activities, including those that we don’t want. Which ones are worse? Who decides?
The clothes that we buy for our children are sewn with cotton farmed, dyed and processed at great ecological cost, and exploiting everyone from the field workers to the shop assistant who sells them to us.
I find the attitude of cherry picking individual products or services and labeling them as evil intellectually very dubious.
