I don't use copilot but Claude integrated in Spacemacs through gptel.
On difficult things (new conceprs I don't already know): -100% efficiency. I just end up reading the manual/code and figure out things by myself.
On day to day things: +20% efficiency, mainly because it's a faster search/reminder.
On simple things (like boilerplate): +300% efficiency. It eliminates the tedious, boring bits for me so I have more time and brainpower to focus on the first category.
How I reached these numbers? gut feeling.
How I get consistent results? I add project files as context and then write as detailed prompt as I possibly and reasonably can (this does take time). Rubbish in => rubbish out, so it's important to prompt well. I've found out that I need multiple successive prompts to get good results with refinments, rewriting in place always gave me poor results.
Overall, it's a boost but since nowadays I tend to spend my time in the first category mainly, I don't end up getting that much out of it. The 10% people claim seems right to me.
I guess I'm more motivated to do the boring bits as I'm not the one actually doing them, just reviewing, and they fly by faster, but how do you even quantify that? enjoyment points?
I think that AI won't replace programmers, at least for now. All those codes written by AI still need review and more attention within security. Here is a good article about Copilot: https://gitprotect.io/blog/github-copilot-introduction-an-ai...
10%-ish is about right. In Fred Brook's No Silver Bullet essay he makes the point that software development has multiple phases, say:
* requirements gathering
* design
* coding
* testing
* deployment
and suppose these all make an equal contribution. If you managed to make a tool that can do the coding in 0% of the time you still have to do 80% of the work!
I've tried to calculate this.
I used copilot for 14 days and every time I used copilot and found it benefits me (saved me time), I wrote a check mark on a sheet of paper. Every time I tried to use it and it led me down a rabbit hole or simply didn't do much, I wrote a minus sign.
In the end there were roughly the same number of checks and minuses. Granted-- it wasn't the best experiment in the world and quite subjective.