2025 0201 Books and Bibliogapher

I’ve been quietly adding books to my website as a parallel taxonomy¹. The Play Hades and read the Iliad post references the most books so far, but there are a few others I’ve added, with more to come.

I started out doing this by hand, but of course its impossible to countenance spending dozens of minutes on a rote task after considering the opportunity to spend dozens of hours on an over-engineered “shortcut”.

tl;dr: I wrote bibliographer, which pulls books in my Audible and Kindle libraries and saves the metadata where Hugo can find it.

Some books

Before we dive into the technical stuff, I’m particularly excited about being able to link to some favorites. Like all posts in the books section, none of these pages will be all that interesting on their own; this is why I don’t expect anyone to follow the section RSS feed. But now I can tag longer blog posts by books when I’m inspired by their ideas.

• Blindsight, hard science fiction abount consciousness that absolutely blew my mind when I first read it. Released under Creative Commons for free directly from the author.
• Free Culture, which deeply influenced my thinking about intellectual property and copyright.
• The Righteous Mind, which gave me a model for understanding people with different values than I have.
• The Revolt of the Public, which gave me a set of tools for understanding why populism is on the rise.
• Neuromancer, the source of one of my favorite quotes.

o1 is really quite good

My intentions of an enjoyable time sink were nearly routed: I tried ChatGPT o1, and it gave me almost 1000 lines of almost working code in one shot. Even with the “almosts”, I was really impressed.

All was not lost, however. It turned out my first instructions took a lot more planning that I normally give to LLM interactions. I tried out the suggestions in o1 isn’t a chat model and it paid off very well. I think it wasn’t quite as fun as jumping right in to write code, but (all jokes aside) the time efficiency was very good, and it drove home the value of planning. You can see my original plan document. I wrote that, including stub functions, before asking ChatGPT anything. I only used o1 is it was available to me in my $20/mo ChatGPT Plus plan; I never used the $200/mo ChatGPT Pro.

I’d love to share the full transcript, but the conversation is large enough that the fucking website doesn’t work. (Isn’t it going to be just tragic if OpenAI et al manage to obsolete the job of programming a computer but every consumer product is broken just as much as it is today?)

Eventually it got big enough that o1 was forgetting things or introducing bugs, and I took a more active hand in writing the code and asked smaller more scoped questions of cheaper models.

Using bibliographer with Hugo

The GitHub repo has instructions for use, and an example snippet to include in a Hugo theme.

It caches API call results inside assets/bibliographer/apicache/*, and creates mapping files under assets/bibliographer/usermaps/* that you can edit if the heuristics that generate a slug, find the ISBN, find the cover, etc don’t return good results.

All bibliographer does is write data filse and empty index.md files; it’s up to your theme to render them how you like, and up to you to write content in the book posts if you want to.

Future upgrades to this site

I’d like to add author and series cross linking for the site as well, so that someone looking at a book page based on a recommendation can easily find related books without me having to write prose explaining the relation.

I haven’t gotten around to this yet, but it’s on the list for when I have extra tinkering time.

Working with Amazon

I was surprised at how difficult it was to get data out of Amazon.

There is no official support for exporting a library from Kindle or Audible in any format whatsoever. Goodreads is owned by Amazon and is privileged with access to a private Kindle API, but it’s not accessible to users. (Oddly, Audible, also owned by Amazon, is not supported by Goodreads at all.) There is also, of course, no public Goodreads API. (I haven’t implemented Goodreads support in bibliograhper, but I would be interested in doing so if there were some way to get the library data and anyone else wanted the feature.)

Amazon uses an internal product ID called an ASIN. Unsurprisingly, it doesn’t publicize any database it might have of ASIN to ISBN, but it also doesn’t have a public database of ISBNs to ASINs. The Amazon Product Advertising API is not public; if they don’t deign to give you access, they suggest clicking around on their website to generate associate links like an animal.

What do we do instead? Well

• The Python audible package is amazing! It has reverse engineered the Audible API and was super easy to use.
• To export your Kindle library, you have to log in to the Kindle webapp and run some JavaScript in the developer console. Dire.
• To get ASINs from ISBNs, we scrape Amazon.com web pages. (Fun fact: I have it on good authority that this is current state of the art for pricing consumer electronics produced by Fortune 500 companies.)

Other cool APIs I found

• The Google Books API is amazing and free
• LibraryThing has some niche APIs that might come in handy in the future
• OpenLibrary APIs are free and completely unauthenticated, but the data there is crowd-sourced and a bit unreliable

Out in the wild

Feature requests and bug reports are welcome. If you end up using it for something, I’d love to hear about it!