How does that work with logs? Logs are often... Huge? How many lines of logs can you paste? Because if I first need to narrow down the log to the problematic part, I kinda already have my problem right there no?
Or do you mean I do something like grab the lines with "error" in the log, hoping there aren't too many, then ask ChatGPT what it thinks about this:
[ 0.135036] kernel: ACPI Error: AE_NOT_FOUND, During name lookup/catalog (20210730/psobject-220)I wish people would focus on these exceptional strengths of the model rather than blabbing about AGI or whatever.
Similarly to you, I have been able to find issues with logs, formatting, asking it quick query questions in [whatever flavor of query language XYZ service likes to use], etc.. and it's really, really good.
The alternative is to muscle through it, using a lot of energy, writing my own parser or something dumb, or to use Google - which basically isn't usable anymore!
But you have people who are like "GPT CODED MY ENTIRE WEBSITE" and "GPT TAUGHT ME QUANTUM PHYSICS" and I'm like... uh... big doubt my man...
I let GPT-4 walk me through troubleshooting steps with my extremely slow read times on my SSD-backed KVM virtual machine. It told me the things it needed, I pasted relevant logs and other output, and finally I solved my issue. I was highly impressed! It parsed atop, top, and various other content, explaining exactly what everything meant.
Another benefit was that it was able to present a much more readable version of some of what I pasted. I may have to start using it for cleaning up hard-to-read output (looking at you, atop!) in the future, it really excels at that!
Also, the issue ended up being that I that I was reading from what turned out to be an NFS mount. Doh!
Can any of the existing open source models do the same?
ChatGPT is great but I don't want all of my queries going to OpenAI.
I'd rather shell out a considerable sum to buy the equipment to run my own.
This is a great feature and I did use it few times to test. However, be aware of potentially leaking your company's private or sensitive information when doing this.
2023-05-04T04:20:42.199366-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Successfully activated service ' org.xfce.Xfconf' 2023-05-04T04:21:18.294427-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T04:21:20.447830-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T04:22:59.419150-04:00 penetrati ngnu systemd[1]: Started session-108.scope - Session 108 of User siebel. 2023-05-04T04:25:01.181760-04:00 penetrati ngnu CRON[363165]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2023-05-04T04:26:48.757845-04:00 penetrati ngnu systemd[1]: session-78.scope: Deactiv ated successfully. 2023-05-04T04:30:42.121839-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Activating service name='org.xfc e.Xfconf' requested by ':1.24' (uid=1000 p id=3360 comm="xfce4-panel") 2023-05-04T04:30:42.239103-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Successfully activated service ' org.xfce.Xfconf' 2023-05-04T04:35:01.251472-04:00 penetrati ngnu CRON[367999]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2023-05-04T04:39:01.202453-04:00 penetrati ngnu CRON[369932]: (root) CMD ( [ -x /usr /lib/php/sessionclean ] && if [ ! -d /run/ systemd/system ]; then /usr/lib/php/sessio nclean; fi) 2023-05-04T04:39:01.267940-04:00 penetrati ngnu systemd[1]: Starting phpsessionclean. service - Clean php session files... 2023-05-04T04:39:01.431025-04:00 penetrati ngnu systemd[1]: phpsessionclean.service: Deactivated successfully. 2023-05-04T04:39:01.431258-04:00 penetrati ngnu systemd[1]: Finished phpsessionclean. service - Clean php session files. 2023-05-04T04:40:42.120741-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Activating service name='org.xfc e.Xfconf' requested by ':1.24' (uid=1000 p id=3360 comm="xfce4-panel") 2023-05-04T04:40:42.214032-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Successfully activated service ' org.xfce.Xfconf' 2023-05-04T04:40:52.623497-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T04:45:01.213527-04:00 penetrati ngnu CRON[372890]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2023-05-04T04:49:37.313682-04:00 penetrati ngnu smartd[746]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Tempera ture_Cel changed from 60 to 61 2023-05-04T04:50:42.117968-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Activating service name='org.xfc e.Xfconf' requested by ':1.24' (uid=1000 p id=3360 comm="xfce4-panel") 2023-05-04T04:50:42.229128-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Successfully activated service ' org.xfce.Xfconf' 2023-05-04T04:54:00.082471-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T04:55:01.262268-04:00 penetrati ngnu CRON[377718]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2023-05-04T05:00:42.119305-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Activating service name='org.xfc e.Xfconf' requested by ':1.24' (uid=1000 p id=3360 comm="xfce4-panel") 2023-05-04T05:00:42.199605-04:00 penetrati ngnu dbus-daemon[1196]: [session uid=1000 pid=1196] Successfully activated service ' org.xfce.Xfconf' 2023-05-04T05:01:53.268834-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:01:55.324225-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:02:02.590368-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:02:05.152270-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:02:10.779523-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:02:18.766764-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:02:41.199587-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:02:50.721657-04:00 penetrati ngnu wpa_supplicant[913]: wlan0: CTRL-EVEN T-BEACON-LOSS 2023-05-04T05:05:01.237788-04:00 penetrati ngnu CRON[382556]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2023-05-04T05:06:43.123423-04:00 penetrati ngnu systemd[1]: Started session-115.scope - Session 115 of User siebel.
┌──(siebel㉿ penetratingnu)-[~] └─$ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������ ������������������������������������������
It truly seems to be the calculator for text.
Yeah it's pretty awesome! I used GPT-4 last week to fix my corrupted SSD. Granted I already narrowed down the kernel logs to a few suspicious lines, but I just pasted those 10 lines in and asked for a fix. Pasting in GPT's arcane `fsck` incantations and boom -- fixed SSD. Saved me an hour or two of hassle reading man pages and stack overflow posts.
Phind.com is so much better.
I just want to pay for the service in exchange of ensuring they won't use my data but I can't find how.
Awesome! It is also fantastic for understanding chunks of Bash script, Perl, AWK etc where if you don't know something then it's next to impossible to search for it using a non-AI search engine.
Bonus: you can also use it to understand what the various flags in a command do.
You better ask it to write software to analyze logs rather than analyze logs. Or your OpenAI bill will go to the roof.
systemd is a very common log.
I'm curious whether you think this would work on logs for custom software that by necessity didn't have either its logs or writing about its logs in the training set.
I used recently to navigate a quite complex Bash script. This is using Codex.
I'd go just below the line that I don't understand and type
# Explain the line above in detail: << gpt explanation >>
However, there was one instance where it almost freaked me out as the output was quite human like:
# Explain the line above: <<I don't understand it>>.Great use case. I have successfully used it in a similar way with SRT files (recording transcripts in SubRip Subtitle format) as well as CSV data from surveys that often has many columns with long text labels (the survey questions) and free text answers.
Ex. a prompt I had used for a developer skills survey .csv was:
> The CSV data below is the results of a skills survey sent to a group of software engineers. The first row is a header row. Please summarize this data in the areas that the People are Strong, Weak, Most Similar, and Unique:
Then, because of things I saw in the response, I asked a few follow-up questions:
> How much Azure experience is there in the group?
> Can you provide more explanation around your assessment that "Engineers generally have little to no experience with Dockers and Kubernetes."
> What other skills and experience do you see in the results that you haven't already mentioned?
To address my risk tolerance vis-a-vis the ChatGPT warnings (and previous UI leak of responses), I replaced the email addresses in the .csv file with "PersonA", "PersonB", ...
Yesterday I dumped a broken markdown table and asked for the first "column" as CSV values. Works really well for this stuff too.
As another side note, using ChatGPT as a search engine is not so great.
Recently, I was trying to find golang libraries for managing authorization. The first one was a well known library (I was incidentally trying to avoid).
But the others 4 were complete were completely fantasy projects. ChatGPT simply invented complete projects with descriptions and github urls.
Interestingly, looking at the urls, it seemed to actually be a composition of authors working on the subject and similar project in other languages.
I also observed the same behavior when I tried to find community resale/recycling centers (ressourcerie in French) in my neighborhood, and sure enough it generated a bunch of fake addresses.
It's logical in hindsight, ChatGPT is a generative AI after all. But these results left me scratching my head at first.
I concur ive been using it daily for all sorts of tasks, cant read Microsofts docs round it up get the important stuff i love it :)
What’s wrong with “grep -i error”? That gets me to the source of the error, usually.
Even free one managed to decipher my 10 years old perl code and I was a bit surprised by it.
But in other instance I pasted same function but with parameter name changed (event -> events) and it just produced lies
Anomaly detection is my current favorite use case for GPTs. Other problems can potentially be recast as anomaly detection that are not currently thought of as such. Regulation for instance.
There's a company that does this log analysis in real time for you called Zebrium
Same for obscure C++ compiler errors. You might need to make a small repro, first.
Even better, give it a sample and have it write regex to find other entries like it.
also for fixing bugs in a chunk of code. at least, that has worked for me a couple of times. it can be frustrating if it's struggling, feels like you're stuck in a loop and it's hard to break out
yes, but theres a limit to the characters input into chatgpt so you cant always just dump the logs in.
How do you deal with the token limit?
wouldn't embeddings be a better than ChatGPT?
[dead]
I used it to transform cryptic credit card statement items into company names, which then allowed me to query my Gmail archives for receipts and invoices from these vendors, automating a manual process of accounting backup discovery that is the bane of my very existence. I even got GPT-4 to assess whether an email likely relates to an invoice or payment so that I could limit the amount of noise extracted from my email archives.
I highly recommend considering GPT-4 every time you encounter a painful manual process. In nearly every case where I have applied GPT-4, it has been successful in one-shot or few-shot solving the problem.