Passacaglia and Fugue

Book Review: Amazon DynamoDB - The Definitive Guide, by Aman Dhingra and Mike Mackay
Over the years I've preferred to find out about new tools and techniques by reading a book about them. For quite a while, I'd been looking for a book on DynamoDB, but none worth taking seriously seemed to show up, as if this mode of learning had gone out of fashion (despite books on other NoSQL databases being readily available). I came across this recent release rather randomly, by searching without much expectation.

This covers the necessary ground, and it does so competently. To me, DynamoDB is difficult to classify among the varieties of NoSQL - is it a key-value store or a document store? It leans more toward the former, but with some features of the latter. The book gives various examples of the operations that can be performed, including through the use of the AWS console and the NoSQL Workbench tool, with a good description of the billing models available.

A fair amount of the book is dedicated to database design, with contrasts drawn from the more conventional normalization of RDBMS schemas. Secondary indexes are explained, with the pros and cons of the local and global options. There's some material on the internal architecture of DynamoDB, which is interesting but perhaps not directly relevant. More advanced topics include backup and restore, change data capture (CDC) streams, caching and multi-region operations. The final chapter on migrations is particularly practical, including some cases where DynamoDB may not be a good option. This book feels like it fills a gap in the market, and gives a pretty good grounding on the database.

Book Review: OAuth 2 in Action, by Justin Richer and Antonio Sanso
I began reading this before going on holiday, but it didn't feel like holiday reading, so I put it down until I returned. Then I had covid, so this was very much a reading in two halves. But it also felt like a book of two halves, with some of the later chapters going a bit further off-topic than I would have liked. The book begins with a thorough run through several OAuth scenarios, showing how the different grant types work and explaining their pros and cons. It's a complicated subject and I'm not sure these are always clear. Several chapters are spent discussing potential implementation vulnerabilities and whilst there is good advice and it's clearly well-intentioned, it's alarming to learn the ways in which such a widely used standard can be abused. The discussions on JWT and OIDC are very useful to my work, but the later chapters on US health care standards seem irrelevant. On the whole I found this a decent reference for the technology, but there are quite a few chapters I doubt I'll dip into again.

Book Review: The Mythical Man-Month - Essays on Software Engineering (Anniversary Edition), by Frederick P Brooks Jr
This classic came up a few months ago at work; whilst I'd heard of it and read the No Silver Bullet paper as an MSc student, I had never read the whole book. I decided it was worth a go, to see how relevant it still was. (The author was a manager working on IBM System/360 and OS/360 in the 1960s/70s, the book originally published in 1975 with this update twenty years later.)

The first thing I noticed was the quite chatty and cheery style; it has the air of fond reminiscence of something achieved, despite difficulties. It also has an air of pragmatism, pointing out the need to work within the reality of any situation. There are some aspects of the book that are laughably dated - who below C-level has secretaries now? - and aspects that are dated in other ways - Brooks exclusively uses male pronouns to refer even to abstract colleagues. (Though to be honest, it grates in more modern books when one feels the author pointedly using female pronouns. I'm sure it's not easy to word everything in a neutral style, but perhaps we should try harder.)

But by and large, and lifting from Brooks's terminology in the No Silver Bullet chapter, the dating of technology is "accidental" rather than "essential" to any of his arguments. On the title essay, the principle is that the conventional relationship between effort (man-months) and delivery timescales does not apply to IT projects: there are practical limits that mean you can't deliver something more quickly by adding more human resources to it, in a way that might apply in other industries. The interrelationships between tasks just generates dependencies and non-linearity. Other essays in the book give thoughts and advice on how to better run such a project.

I found the essay on "Surgical Teams" particularly interesting, as it is perhaps the one area where the author's views do not conform to current practice - he argues for a strong hierarchical and command structure, whereas current practice emphasises interchangeability and flattened, empowered teams. However, the corollary of this is that it undoubtedly weakens what Brooks calls "conceptual integrity": in almost all services and systems the absence of a singular "controlling mind" is visible in the inconsistencies and joins between components, or even individual endpoints within a service.

Whilst it's not written in such terms, there is clearly embryonic thinking about agile processes. In one of the update chapters, Brooks reflects on, and has the honesty to disagree with, some of what he had written earlier. The waterfall model comes in for some criticism and he re-evaluates his approach to prototyping with an emergent direction towards incremental delivery. This perhaps is an "essential" change, the ability to release software more or less on-demand.

But fundamentally, Brooks's Law still applies: Adding more software developers to a late-running project makes it run even later.

Book Review: Hacking with Spring Boot 2.3 - Reactive Edition, by Greg L Turnquist
I've been working with the Spring Webflux reactive stack for 18 months or so now, and whilst it does make more sense, I still feel it has a very steep learning curve without enough learning materials. Having seen a presentation by Josh Long, I didn't feel confident his book would help with real-world scenarios, and a colleague who has a copy gave it only a lukewarm endorsement; I came across this one and despite the somewhat off-putting title, decided to give it a go.

It's not a bad book, and I did learn some things along the way, but I think it may suffer from the author writing two books in parallel (there's a similar title for the "Classic Edition", which presumably deals with the conventional web stack). To be fair, the book is about Spring Boot; quite how much of the Spring ecosystem should be drawn into scope is debatable. Turnquist reaches into Spring Data, Spring Boot Actuator, and Spring Boot Security, which is a good start. Despite being one of the contributors for Spring HATEOAS, he unfortunately alights on some of the limitations and disadvantages of this format in a chapter on RESTful APIs, offering rather quirky workarounds. But on the specific reactive stack, which is the differentiating part I was most interested in, there are somewhat slim pickings. The basic idea behind reactive programming is outlined, along with the base APIs, but it rarely goes anywhere beyond superficial. There is no good explanation for the difference between map and flatMap, with little more than advice to try the latter if the former doesn't work. Yet this is a fairly fundamental point, about which I am hesitant, and still little the wiser. It is something to do with the Reactor framework kicking off another subscription, I think, but it is not well explained (and had I found good explanations on the web I wouldn't have bought this book). R2DBC is mentioned but avoided as not yet mature at the time of writing (publication was May 2020), with little suggestion for how to face the almost inevitable real-world problem of melding reactive and non-reactive code.

The code examples, sadly, are all too reminiscent of real-world code, often difficult to read. Through my career we have progressed from DLL hell through Classpath hell and now into Lambda hell. If I have to fight with counting parentheses and indentations when reading, then it's poorly-written rather than idiomatic code IMO. On the other hand, the sections on testing were more useful, with reasonable efforts around StepVerifier and a useful introduction to BlockHound. On the whole this book is probably a reasonable short introduction to Spring Boot, but someone already familiar with it will have less to gain.

Book Review: Domain-Driven Design - Tackling Complexity in the Heart of Software, by Eric Evans
I'd been aware of this book for quite a while, and when it was heavily reduced on Kindle I decided the time was right. I wondered whether it would reveal that I'd been doing it all wrong for years, or whether I was doing it right without attaching the right names to things.

It's a mixed book. I found the early discussion on entities, value objects, and repositories, illuminating in that they clarified what I already intuitively understood. But whilst there was nothing to object to in the later parts of the book, it didn't really retain my attention. Published in 2004, it hasn't aged well. This may be a reflection on my personal technology stack. The code examples in the GoF Design Patterns are generally written in C++ or Smalltalk, and so were never all that familiar to me. Here, the code examples are in Java - and it's Java without generics, it's Java with entity beans, that's how old it is.

There's a whiff of Conway's Law to it as well. Design choices can be political and organisational, as well as technical. Here one senses the stale bureaucracy of enterprise software, which certainly isn't dead, but is something I suggest most enthusiastic techies would tend to avoid in a way that perhaps wasn't possible a decade or more ago. Enough companies have woken up to the need to devolve control, and to get success from that. The concepts are still valid, but the way they are presented does not always feel all that relevant.

Book Review: Normal Accidents - Living with High-Risk Technologies, by Charles Perrow
I came across this book from a blog post I encountered on my feed. It chimed with problems both technical and cultural at work: the idea that increasing safety measures does not always make a system safer. Of course, I'm a software engineer and the system isn't a hospital or a nuclear power station, but I was interested to see whether there were any principles that could be transferred.

It's quite a dry, academic read, but not impenetrably so. Perrow begins with an example of a simple mistake one might make in the morning, and a series of incidents that may follow from it on the way to a job interview. The sequence of events sounds unlikely, but that is rather the point. There follows a more thorough discussion of the Three Mile Island accident of 1979, of which I have a vague memory as a child. Establishing cause is one thing; establishing responsibilities is another. A catalogue of errors and neglect emerges. The point is, most of these things, on their own, would not have led to the accident: the unlikely coincidence of these things made it inevitable. Further chapters examine other industries, such as the chemical industry, air and marine transport, dams, mining, space travel and genetic engineering. The results are rarely reassuring; the nuclear industry gets highlighted only because it is relatively new, and can, in the case of leakage, readily cause damage to a much wider environment. Only air travel emerges with a reasonable bill of health, based on a combination of self-interest (no airline wants a bad safety reputation) and a culture of "blameless" reporting systems (though one never knows how blameless it actually is) for minor inconsistencies and infringements.

This is a social science book, and Perrow probes the convenient tendency for investigations to assign "operator error" as the cause of accidents. While this can't be excluded, he argues there are other factors at work. His thesis involves two axes, one of system complexity, and the other of system coupling, giving a diagram of four quadrants.

In the high-complexity, high-risk quadrant, he argues, accidents are inevitable (hence "normal"), because there is no appropriate management strategy: a strong command-and-control structure isn't flexible to allow discretion for operators to tackle an incident creatively, but high coupling between system parts and a lack of slack in the system reduces the scope for autonomy. This is where the arguments in the book become relevant to a discipline such as software engineering. For me, it's not because our systems land in a particular quadrant on Perrow's chart, but rather that we have the opportunity to choose (at least to some extent) where on the chart to build our systems. Reducing complexity and coupling is not always easy, unfortunately it sometimes doesn't even get considered and it can be a difficult "sell" to management for adding in to an existing product, even after an accident; but putting in more and more rules and "process theatre" may not make a system safer (and is guaranteed to make it less responsive). Perrow documents several cases where safety systems failed or made an accident worse.

I read the 1999 edition, which concludes with an additional chapter on some major accidents since the first edition, including the Bhopal chemical plant, Chernobyl and the Challenger space shuttle disasters. Fortunately his final chapter, on predictions for the Y2K bug, turned out to be way too pessimistic, but even he acknowledges this event was something the industry knew was coming (even if it only dealt with it at the last minute), which is a different characteristic from most of the other events described in the book.

Book Review: Accelerate - Building and Scaling High-Performing Technology Organizations, by Nicole Forsgren, Jez Humble and Gene Kim
This was the book all the speakers were talking about at QCon. Of course they were, by and large, following the practices advocated, at which point maybe it's something of a post hoc justification; but maybe there was a greater variety of positions among the attendees at large, and it would give them some pause for thought and/or evidence for advocacy in changing their practices back in the office. Way back in the 1980s Fred Brooks posited that there was no silver bullet for software engineering; yet in recent years, DevOps has really taken off, and the authors have data that suggests adoption of some key practices - measured by deployment pipeline duration, deployment frequency, mean time to fix, and change failure rate - really does distinguish high performing organisations from the rest, even to the point of possibly defining an inverse Conway manoeuvre. In the first part of the book, they offer some commentary and suggestions for taking this approach. The second part takes an unexpected tangent, discussing the merits and designs of survey-based research; I'm sure this is valid stuff, but it seems to target quite a different audience, of survey researchers rather than IT practitioners.

Book Review: Building Microservices, by Sam Newman
This jumped the queue, on loan from a former colleague. Prompted by some of my quibbles in the new job, I wanted to check on received wisdom and, if there is such a thing, best practice. Although this book dates from 2015 I don't think it is out-of-date yet (there is a second edition in the pipeline, currently about a year off). Rather than being about a specific technology or implementation, this book is about microservice architectures and patterns. Sensibly it begins with the benefits of microservices, and the difficulties with monolithic systems that drive their adoption. There is also some more controversial discussion of topics where microservices can challenge conventional wisdom, such as the provision of client-side libraries or the use of DRY principles. (Strictly, I think DRY is a good principle, but it has limits, and it's bad if it leads to over-coupling between things that should be allowed to move independently.) There are chapters on looking for ways to slice a monolithic application, microservice deployment, testing, integration, monitoring, security and scaling. Sometimes I felt it could have done better in the way of examples, but that would have tied it to implementations. The problem of replacing database-level integration with cross-service joins doesn't seem to be examined all that well, as it's easily a performance nightmare; some cases, it's true, can be transferred appropriately to a data warehouse solution, but perhaps there could have been more advice on dealing with joins for synchronous user requests. But for the most part, without going too deep, it is quite thorough.