My random advice from the internet.
Prototype at the level of text output at the shell level bash/sed/gawk/grep because:
1. It may turn out that that's all that's needed.
2. Making changes in one place is easier.
3. It is more flexible when trying to figure out what should be done.
4. It will not introduce a bug into the existing code.
5. It will not create a dependency between the components.
6. The shell is the appropriate level of abstraction for logs.
This might be a good question for Software Engineering on StackExchange.
Good luck.
My random advice from the internet.
Prototype at the level of text output at the shell level bash/sed/gawk/grep because:
1. It may turn out that that's all that's needed.
2. Making changes in one place is easier.
3. It is more flexible when trying to figure out what should be done.
4. It will not introduce a bug into the existing code.
5. It will not create a dependency between the components.
6. The shell is the appropriate level of abstraction for logs.
This might be a good question for Software Engineering on StackExchange.
Good luck.