In the last post, we did an LLM prompting experiment to build a Google Calendar integration inside Ruoom’s open source CRM code.
It worked (!) but before we get too excited, we need to do a code review.
Why? 🤔
Because in software development, “it runs” isn’t the same as “it’s reliable.”
Code reviews are where performance, security, and maintainability get validated; the foundation of software quality assurance. They help catch architectural drift, logic errors, and inconsistencies that slip past even the most advanced AI. Without this step, it’s easy for technical debt to accumulate. Functions overlap, naming conventions drift, and assumptions about data handling go unchecked.
A proper review lets us:
- Verify the code follows our internal style and design patterns.
- Check how it handles edge cases and failures.
- Confirm it integrates cleanly with the rest of the system.
- Identify where automated tests (if any) fall short.
This is the step that turns “AI-generated” into “production-ready.”
So here’s our human team and our human brains using no robots to do a code review.
No code review automation here.
Anyway, let’s get into it.
Reviewing the Code
You can follow along with this process by peeking at our public repository.
We’ve created a branch specifically for the code from this blog series: Ruoom/ruoom-core at llm_experiment/gcal_import
Using Git, we can isolate only the changes made during our previous blog. We’ve grouped them into a single commit labeled “LLM-generated feature code”: LLM-generated feature code · Ruoom/ruoom-core@cc66f3b
Let’s start by reviewing the files that have changed:
We can group the changes into 3 categories:
- Files which the LLM edited to add its requisite functions:
- schedule.js
- schedule.html
- administration/urls.py
- registration/models.py
- Files which the LLM wrote from scratch:
- gcal.py
- Supporting files:
- Requirements.txt
- settings.py
- 0004_profile_gcal_fields.py
Directive #1 is to ensure that the existing code hasn’t been disturbed.
Reviewing the first 4 files should enable us to do so. If there is an existing test suite, this is where we would run it to confirm core code performance.
gcal.py being a completely new file, we need to closely examine it for consistency with the rest of the repository.
The supporting files will only require a short review as they reflect the changes made to the other files.
schedule.js
Let’s start at the top with schedule.js – the javascript code which populates the front-end calendar with events:
There are only 2 significant changes:
- The LLM removed a number of lines which it didn’t think were needed for this function
- These lines are here for compatibility with schedule plugins we offer. While the LLM wouldn’t know this, it’s strange it would simply delete code it doesn’t think is needed. Every developer knows that deleting code without triple-checking its purpose is a recipe for disaster!
In the Kdrama, Startup, the main character Nam Do-San deletes code with no backup or version control (at least from what we can see on screen), and we have not stopped talking about this scene for years. He straight up does a select all, delete. Don’t do this. Please.
- In fact, when testing the front end calendar, we discover that deleting this code has broken the alternate “Day” and “Week” views of the calendar. Ugh.
- The LLM added logic to insert Google calendar events onto the FullCalendar implementation (FullCalendar is an open source calendar solution we use)
- Our first observation is that the LLM is making use of a feature of the FullCalendar module called “eventSources” which our developers hadn’t used before. It’s interesting that the LLM has researched FullCalendar usage/documentation (eventSources – Docs | FullCalendar) to make an implementation decision.
- The feature implementation is elegant in its presentation, if perhaps too condensed. The LLM placed a function definition directly within an array. We don’t hate it, but some developers might.
schedule.html
Reviewing schedule.html, the only change is the addition of the “Connect/Disconnect Google Calendar” buttons. The implementation of which button appears is straightforward:
We have 3 comments about its implementation:
- It’s good to see the LLM is using off-the-shelf button/badge styles for these buttons. However neither the LLM nor we are UX designers and this needs to go through a UI/UX review process (this sounds like another blog post 👀).
- During our previous blog, we had an issue where clicking “Connect Google Calendar” didn’t work because we hadn’t properly configured credentials. Again, from a UX perspective, we had zero feedback regarding where the issue was. It could have been a server error, an issue with my Google Cloud Console, or a typo in my credentials. In a revision we would want feedback on the success or failure of the calendar sync presented to the user.
- We may need to expand the number of states or messages presented to the user. For instance, if the user has to re-authenticate with Google due an expiring login session.
administration/urls.py
The changes to the urls handler are straightforward: the LLM added 4 endpoints for the front end to call. No comments on the urls.py file itself as we will review the back-end logic a little later.
registration/models.py
The LLM’s database implementation is a little more interesting:
First, we observe that once again the LLM is deleting code. Why?!
In fact, this time, the LLM deleted code that was very much necessary.
The first_last_user() function which presents a user’s name in as Firstname Lastname is called by the front-end and breaks the front end if deleted. We had to manually add this function back into the code to get it to work during the previous blog (which is why it’s also reflected as added code on the right side of the screenshot).
On the flip side, the decision the LLM made to associate the google sync/credentials at the user level instead of the system level is a good one. Maybe this is obvious, but we or a developer might have assumed that the requirement was to link google calendar to the entire system. As implemented here, each user logging into the CRM will see their own calendar reflected on the page not only providing a tailored experience but also preventing privacy concerns whereby other users may spy on someone’s Google calendar.
Finally, you might notice the new “google_credentials_json” property in the user profile object, which represents Google credentials that are not encrypted!!
We are absolutely judging the LLM right now.
We shouldn’t have to explain to you, or to the LLM why this is dangerous but we will.
Depending on the scope that we have set within the Google Console OAuth app, someone might be able to use these credentials to access Google applications beyond the Calendar (Access to your Gmail gives someone freedom to start resetting all of your passwords).
Bishop Bullwinkle knows what’s up.
A compromised database, SQL injection, or simply a bug in the code could all expose this data, and it certainly violates data security standards.
These plaintext credentials must be encrypted at rest.
gcal.py
This entire file is a new creation, so we have to go through it carefully.
Functions defined in this new file:
Supporting Functions:
- _get_google_client_config()
- _serialize_credentials()
- _credentials_from_json()
These first 3 supporting functions are well-designed and structured with good conventions. No notes.
OAuth Functions:
- google_oauth_start()
- google_oauth_callback()
- google_oauth_disconnect()
To start, there are maybe 2 comments in this entire file.
We need significantly more comments, especially when the code is dense and not self-documenting. We don’t even know what the purpose of each function is. If this code is to be maintained in the future, engineers need to quickly understand what it does & how.
There are also a few instances where the LLM throws “import” statements into the middle of functions. Maybe this is personal preference, but imports should go at the top of the file unless there is a very good reason to handle them in-line.
Error handling does seem to be sufficient, which is great to see. We don’t have much concern over the scalability of the OAuth layer of these functions.
With all this said, we might have liked to see these functions grouped as methods of a Google OAuth class instead of hanging around loosely as they are.
Event Population
- google_events()
This is the most complex function in the file, and yet is written very densely with almost no comments. Again! Remember, no human will be able to explain to you what’s happening here when you need to update it in 6 months. Comment comment comment.
It doesn’t match all of the project conventions around structuring a GET endpoint function, but it’s acceptable.
It makes use of a “while True” loop as it queries for page after page of Google events. We are always suspicious of infinite loops like this – though in this implementation it seems okay as any REST API failure to Google would cause it to break. Still, there may be a safer way to implement this.
The events are returned to the front end as one giant JSON response.
Would this scale to handle thousands of events?
We aren’t sure that it will, but that level of scale hopefully isn’t necessary for Google Calendar.
Conclusions
Overall, the LLM hasn’t done a bad job. We would be happy with this as the output of a junior developer, though it definitely requires improvements. We didn’t ask the LLM to create a test suite but that would absolutely be a next step here.
NOTE: Having said this, we also believe it’s vitally important to continue to hire human junior developers in the age of AI so we can train the next generation.
In our evaluation of the code, we’re going to hit many tired refrains of developer reviewers around the world:
- More comments!
- Don’t delete code you don’t understand!
- Encrypt your credentials when they’re at rest!
- Don’t let engineers design our UX!
- Reinforce our coding conventions!
- Write tests!
Many of these issues can probably be ironed out quickly with a few more iterations with the LLM. We hope this doesn’t introduce even more issues of these kinds, but LLMs are famous for going in circles when asked to fix their own code…
To close this blog as we did the last one, we’d like to reference a reddit post from user u/TreeTopologyTroubado who describes how to “vibe code” at a major software engineering firm:
How we vibe code at a FAANG. : r/vibecoding
You’ll notice that the poster focuses much more on “AI assistance for productivity” than on handing over any sort of design authority or uncontrolled development to the LLM.
We wholeheartedly agree.
LLMs will increase production velocity, but only when placed inside a robust software development process to being with.
In the next blog we’ll run another LLM prompting experiment where we’ll export calendar events from Ruoom’s CRM calendar into Google.
Until then, keep using your brain and don’t outsource all of it to AI.
Talk to you later. 👋