Projects: Category Archive

Posts about software development projects from or maintained by Bit Badger Solutions

Tuesday, March 26, 2024
  myWebLog v2.1

myWebLog v2.1 is now available. There are some great new features in this release.

  • Full podcast episode chapter support is now available. Chapters can be created using a graphical interface, and will be served for applicable episodes by adding ?chapters to the URL of the post for that episode. The documentation has a detailed description of this feature.
  • Redirect rules can now be specified within myWebLog. While it has always supported prior links for pages and posts, this allows arbitrary rules, such as pages that direct to other sites, maintaining category archive links, etc. Its documentation also explains all about the feature and its options.
  • Canonical domains can now be enforced within myWebLog. Adding the configuration for BitBadger.AspNetCore.CanonicalDomains will help enforce the use of www. (or absence of it), as an example.
  • Docker images can now be built within the source. The plan for 2.1 was to provide those images from the outset, but rather than relying on an external registry, we plan to stand up our own for distribution of public container images. If no code changes are required in myWebLog before that registry is available, we will release v2.1 images with this current build; if not, we will do a point release for them.
  • The version of htmx injected for the “auto htmx” functionality has been updated to v1.9.11.

In addition to these features, a decent amount of the development of this version included full integration tests with all three data storage backends. SQLite, PostgreSQL, and RethinkDB are all verified to give the same results for all data operations. (NOTE: SQLite users will need to back up using v2, then restore to an empty database using v2.1; this will update the data representation used in several tables.)

Finally, there are downloads for this release that target .NET 8, 7, and 6 - all the currently-supported versions of the .NET runtime. (The Docker images target .NET 8, which does not matter because, well, Docker.)

Head on over to the release page to get the binaries for your system! Feel free to participate in the project over on GitHub.

(Note: the link above now points to v2.1.1, which fixed an issue with PostgreSQL upgrades between v2 and v2.1. Upgraders from v2 can safely (and are encouraged to) go straight to 2.1.1.)

Categorized under , , , ,
Tagged , , , , , , , , , , , ,

Thursday, November 24, 2022
  Fragment Rendering in Giraffe View Engine

A few months back, Carson Gross, author of the outstanding htmx library, wrote an essay entitled “Template Fragments”; if you are unfamiliar with the concept, read that first.

The goal is to allow an application to render a partial view without defining that partial view as its own template, in its own file. A common pattern in hypermedia-driven applications (and listed “con” for them) is that every little piece that may need to be rendered on its own must be broken out into its own template. This pattern eliminates the “locality of behavior” benefits; you cannot even see your entire template together in one place. Fragment rendering eliminates that requirement.

Developing with Giraffe View Engine, we can do part of this out-of-the-box, as we can control the functions we use to create the nodes that will eventually be rendered. And, while there is not a concept of a “template fragment” in Giraffe View Engine, the term “fragment” as applied to an HTML document identifies its id attribute.

Giraffe.ViewEngine.Htmx, as of v1.8.4, now supports rendering these fragments based on the id attribute. Giraffe has the RenderView module where its rendering functions are defined; opening the Giraffe.ViewEngine.Htmx namespace exposes the RenderFragment module. Its functions follow the pattern of Giraffe View Engine:

  • AsString functions return a string
  • AsBytes functions return a byte array
  • IntoStringBuilder functions render into the provided StringBuilder

Each of those three modules has htmlFromNodes if the source is a node list, and htmlFromNode if the source is a single node. (As fragment rendering has little-to-no use in the XML world, XML rendering functions are not provided.)

One final thought - while this was built with htmx applications in mind, the concept does not require htmx. If you want to render fragments with Giraffe View Engine, using this package will allow that, whether your application uses htmx (you probably should!) or not.

Categorized under , ,
Tagged , , , ,

Friday, July 29, 2022
  myWebLog v2

Today, we released myWebLog version 2 (release candidate 1). This project has been roughly 8 months in the making, replacing the 6-year-old v1 with a completely new application.

v2 supports all of the things v1 supported - revisions, prior permalink redirection, etc. - as well as posts and pages written in Markdown, podcast generation, uploads, and administrative actions, all from within the application. Additionally, it supports both RethinkDB and SQLite for its data storage; using the latter, it will run without any further configuration.

There is a lot to be said about it, and most of it is in the documentation; take a look at it. There are a good many features you would expect, and some that you may not; I'll include a few highlights here.

  • Existing support for several Podcasting 2.0 elements, with plans to incorporate even more
  • Theming via a collection of Liquid templates
  • Out-of-the-box support for RSS feeds overall, by category, or by tag
  • Multiple web logs can be hosted by one installed instance
  • A lightning-fast admin interface, powered by htmx

The bits are available on its release page; happy blogging!

Categorized under , , ,
Tagged , , , ,

Tuesday, December 7, 2021
  A Tour of myPrayerJournal v3: Conclusion

NOTE: This is the final post in a series; see the introduction for information on requirements and links to other posts in the series.

We've gone in depth on several different aspects of this application and the technologies it uses. Now, let's zoom out and look at some big-picture lessons learned.

What I Liked

Generally speaking, I liked everything. That does not make for a very informative post, though, so here are a few things that worked really well.

Simplification Via htmx

One of the key concepts in a Representational State Transfer (REST) API is that of Hypermedia as the Engine of Application State (HATEOAS). In short, this means that the state of an application is held within the hypermedia that is exchanged between client and server; and, in practice, the server is responsible for altering that state. This is completely different from the JSON API / JavaScript framework model, even if they use GET, POST, PUT, and PATCH properly.

(This is a near over-simplification; the paper that initially proposed these concepts – in much, much more detail – earned its author a doctoral degree.)

The simplicity of this model is great; and, when I say “simplicity,” I am speaking of a lack of complexity, not a naïveté of approach. I was able to remove a large amount of complexity and synchronization from the client/server interactions between myPrayerJournal v2 and v3. State management used to be the most complex part of the application. Now, the most complex part is the HTML rendering; since that is what controls the state, though, this makes sense. I have 25 years of experience writing HTML, and even at its most complex, it simply is not.

LiteDB

This was a very simple application - and, despite its being open for any user with a Google or Microsoft account, I have been the only regular user of the application. LiteDB's setup was easy, implementation was easy, and it performs really well. I suspect this would be the case with many concurrent users. If the application were to grow, and I find that my suspicion was not borne out by reality, I could create a database file per user, and back up the data directory instead of a specific file. As with htmx, the lack of complexity makes the application easily maintainable.

What I Learned

Throughout this entire series of posts, most of the content would fall under this heading. There are a few things that did not fit into those posts, though.

htmx Support in .NET

I developed Giraffe.Htmx as a part of this effort, and mentioned that I became aware of htmx on an episode of .NET Rocks!. The project I developed is very F#-centric, and uses features of the language that are not exposed in C# or VB.NET. However, there are two packages that work with the standard ASP.NET Core paradigm. Htmx provides server-side support for the htmx request and response headers, similar to Giraffe.Htmx, and Htmx.TagHelpers contains tag helpers for use in Razor, similar to what Giraffe.ViewEngine.Htmx does for Giraffe View Engine. Both are written by Khalid Abuhakmeh, a developer advocate at JetBrains (which generously licensed their tools to this project, and produces the best developer font ever).

While I did not use these projects, I did look at the source, and they look good. Open source libraries go from good to great by people using them, then providing constructive feedback (and pull requests, if you are able).

Write about Your Code

Yes, I'm cheating a bit with this one, as it was one of the takeaways from the v1 tour, but it's still true. Writing about your code has several benefits:

  • You understand your code more fully.
  • Others can see not just the code you wrote, but understand the thought process behind it.
  • Readers can provide you feedback. (This may not always seem helpful; regardless of its tone, though, thinking through whether the point of their critique is justified can help you learn.)

And, really, knowledge sharing is what makes the open-source ecosystem work. Closed / proprietary projects have their place, but if you do something interesting, write about it!

What Could Be Better

Dove-tailing from the previous section, writing can also help you think through your code; if you try to explain it, and and have trouble, that should serve as a warning that there are improvements to be had. These are the areas where this project has room to get better.

Deferred Features

There were 2 changes I had originally planned for myPrayerJournal v3 that did not get accomplished. One is a new “pray through the requests” view, with a distraction-free next-card-up presentation. The other is that updating requests sends them to the bottom of the list, even if they have not been marked as prayed; this will require calculating a separate “last prayed” date instead of using the “as of” date from the latest history entry.

The migration introduced a third deferred change. When v1/v2 ran in the browser, the dates and times were displayed in the user's local timezone. With the HTML being generated on the server, though, dates and times are now displayed in UTC. The purpose of the application is to focus the user's attention on their prayer requests, not to make them have to do timezone math in their head! htmx has an hx-headers attribute that specifies headers to pass along with the request; I plan to use a JavaScript call to set a header on the body tag when a full page loads (hx-headers is inherited), then use that timezone to adjust it back to the user's current timezone.

That LiteDB Mapping

I did a good bit of tap-dancing in the LiteDB data model and mapping descriptions, mildly defending the design decisions I had made there. The recurrence should be designed differently, and there should be individual type mappings rather than mapping the entire document. Yes, it worked for my purpose, and this project was more about Vue to htmx than ensuring a complete F#-to-LiteDB mapping of domain types. As I implement the features above, though, I believe I will end up fixing those issues as well.


Thank you for joining me on this tour; I hope it has been enjoyable, and maybe even educational.

Categorized under , , , , ,
Tagged , , , , , , , , , , , , , , , , , ,

Wednesday, December 1, 2021
  A Tour of myPrayerJournal v3: The Data Store

NOTE: This is the fourth post in a series; see the introduction for information on requirements and links to other posts in the series.

myPrayerJournal v1 used PostgreSQL with Entity Framework Core for its backing store (which had a stop on the v1 tour). v2 used RavenDB, and while I didn't write a tour of it, you can see the data access logic if you'd like. Let's take a look at the technology we used for v3.

About LiteDB

LiteDB is a single-file, in-process database, similar to SQLite. It uses a document model for its data store, storing Plain Old CLR Objects (POCOs) as Binary JSON (BSON) documents in its file. It supports cross-collection references, customizable mappings, different access modes, and transactions. It allows documents to be queried via LINQ syntax or via its own SQL-like language.

As I mentioned in the introduction, I picked it up for another project, and really enjoyed the experience. Its configuration could not be easier – the connection string is literally a path and file name – and it had good performance as well. The way it locks its database file, I can copy it while the application is up, which is great for backups. It was definitely a good choice for this project.

The Domain Model

When I converted from PostgreSQL to RavenDB, the data structure ended up with one document per request; the history log and notes were stored as F# lists (arrays in JSON) within that single document. RavenDB supports indexes which can hold calculated values, so I had made an index that had the latest request text, and the latest time an action was taken on a request. When v2 displayed any list of requests, I queried the index, and got the calculated fields for free.

The model for v3 is very similar.

/// Request is the identifying record for a prayer request
[<CLIMutable; NoComparison; NoEquality>]
type Request = {
  /// The ID of the request
  id           : RequestId
  /// The time this request was initially entered
  enteredOn    : Instant
  /// The ID of the user to whom this request belongs ("sub" from the JWT)
  userId       : UserId
  /// The time at which this request should reappear in the user's journal by manual user choice
  snoozedUntil : Instant
  /// The time at which this request should reappear in the user's journal by recurrence
  showAfter    : Instant
  /// The type of recurrence for this request
  recurType    : Recurrence
  /// How many of the recurrence intervals should occur between appearances in the journal
  recurCount   : int16
  /// The history entries for this request
  history      : History list
  /// The notes for this request
  notes        : Note list
  }

A few notes would probably be good here:

  • The CLIMutable attribute allows this non-nullable record type to be null, and generates a zero-argument constructor that reflection-based processes can use to create an instance. Both of these are needed to interface with a C#-oriented data layer.
  • By default, F# creates comparison and equality implementations for record types. This type, though, is a simple data transfer object, so the NoEquality and NoComparison attributes prevent these from being generated.
  • Though not shown here, History has an “as-of” date/time, an action that was taken, and an optional request text field; Note has the same thing, minus the action but requiring the text field.

Customizing the POCO Mapping

If you look at the fields in the Request type above, you'll spot exactly one primitive data type (int16). Instant comes from NodaTime, but the remainder are custom types. These are POCOs, but not your typical POCOs; by tweaking the mappings, we can get a much more efficient BSON representation.

Discriminated Unions

F# supports discriminated unions (DUs), which can be used in different ways to construct a domain model in such a way that an invalid state cannot be represented (TL;DR - “make invalid states unrepresentable”). One way of doing this is via the single-case DU:

/// The identifier of a user (the "sub" part of the JWT)
type UserId =
  | UserId of string

Requests are associated with the user, via the sub field in the JWT received from Auth0. That field is a string; but, in the handler that retrieves this from the Authorization header, it is returned as UserId [sub-value]. In this way, that string cannot be confused with any other string (such as a note, or a prayer request).

Another way DUs can be used is to generate enum-like types, where each item is its own type:

/// How frequently a request should reappear after it is marked "Prayed"
type Recurrence =
  | Immediate
  | Hours
  | Days
  | Weeks

Here, these four values will refer to a recurrence, and it will take no others. This barely scratches the surface on DUs, but it should give you enough familiarity with them so that the rest of this makes sense.

For the F#-fluent - you may be asking “Why didn't he define this with Hours of int16, Days of int16, etc. instead of putting the number in Request separate from the type?” The answer is a combination of evolution – this is the way it worked in v1 – and convenience. I very well could have done it that way, and probably should at some point.

Converting These Types in myPrayerJournal v2

F# does an excellent job of transparently representing DUs, Option types, and others to F# code, while their underlying implementation is a CLR type; however, when they are serialized using traditional reflection-based serializers, the normally-transparent properties appear in the output. RavenDB (and Giraffe, when v1 was developed) uses JSON.NET for its serialization, so it was easy to write a converter for the UserId type:

/// JSON converter for user IDs
type UserIdJsonConverter () =
  inherit JsonConverter<UserId> ()
  override __.WriteJson(writer : JsonWriter, value : UserId, _ : JsonSerializer) =
    (UserId.toString >> writer.WriteValue) value
  override __.ReadJson(reader: JsonReader, _ : Type, _ : UserId, _ : bool, _ : JsonSerializer) =
    (string >> UserId) reader.Value

Without this converter, a property “x”, with a user ID value of “abc”, would be serialized as:

{ "x": { "Case": "UserId", "Value": "abc" } }

With this converter, though, the same structure would be:

{ "x": "abc" }

For a database where you are querying on a value, or a JSON-consuming front end web framework, the latter is definitely what you want.

Converting These Types in myPrayerJournal v3

With all of the above being said – LiteDB does not use JSON.NET; it uses its own custom BsonMapper class. This means that the conversions for these types would need to change. LiteDB does support creating mappings for custom types, though, so this task looked to be a simple conversion task. As I got into it, though, I realized that nearly every field I was using needed some type of conversion. So, rather than create converters for each different type, I created one for the document as a whole.

It was surprisingly straightforward, once I figured out the types! Here are the functions to convert the request type to its BSON equivalent, and back:

/// Map a request to its BSON representation
let requestToBson req : BsonValue =
  let doc = BsonDocument ()
  doc["_id"]          <- RequestId.toString req.id
  doc["enteredOn"]    <- req.enteredOn.ToUnixTimeMilliseconds ()
  doc["userId"]       <- UserId.toString req.userId
  doc["snoozedUntil"] <- req.snoozedUntil.ToUnixTimeMilliseconds ()
  doc["showAfter"]    <- req.showAfter.ToUnixTimeMilliseconds ()
  doc["recurType"]    <- Recurrence.toString req.recurType
  doc["recurCount"]   <- BsonValue req.recurCount
  doc["history"]      <- BsonArray (req.history |> List.map historyToBson |> Seq.ofList)
  doc["notes"]        <- BsonArray (req.notes   |> List.map noteToBson    |> Seq.ofList)
  upcast doc
  
/// Map a BSON document to a request
let requestFromBson (doc : BsonValue) =
  { id           = RequestId.ofString doc["_id"].AsString
    enteredOn    = Instant.FromUnixTimeMilliseconds doc["enteredOn"].AsInt64
    userId       = UserId doc["userId"].AsString
    snoozedUntil = Instant.FromUnixTimeMilliseconds doc["snoozedUntil"].AsInt64
    showAfter    = Instant.FromUnixTimeMilliseconds doc["showAfter"].AsInt64
    recurType    = Recurrence.ofString doc["recurType"].AsString
    recurCount   = int16 doc["recurCount"].AsInt32
    history      = doc["history"].AsArray |> Seq.map historyFromBson |> List.ofSeq
    notes        = doc["notes"].AsArray   |> Seq.map noteFromBson    |> List.ofSeq
    }

Each of these round-trips as the same value; line 6 (doc["userId"]) stores the string representation of the user ID, while line 19 (userId =) creates a strongly-typed UserId from the string stored in database.

The downside to this technique is that LINQ won't work; passing a UserId would look for the default serialized version, not the simplified string version. This is not a show-stopper, though, especially for such a small application as this. If I had wanted to use LINQ for queries, I would have written several type-specific converters instead.

Querying the Data

In v2, there were two different types; Request was what was stored in the database, and JournalRequest was the type that included the calculated fields included in the index. This conversion came into the application; ofRequestFull is a function that performs the calculations, and returns an item which has full history and notes, while ofRequestLite does the same thing without the history and notes lists.

With that knowledge, here is the function that retrieves the user's current journal:

/// Retrieve the user's current journal
let journalByUserId userId (db : LiteDatabase) = backgroundTask {
  let! jrnl = db.requests.Find (Query.EQ ("userId", UserId.toString userId)) |> toListAsync
  return
    jrnl
    |> Seq.map JournalRequest.ofRequestLite
    |> Seq.filter (fun it -> it.lastStatus <> Answered)
    |> Seq.sortBy (fun it -> it.asOf)
    |> List.ofSeq
  }

Line 3 contains the LiteDB query; when it is done, jrnl has the type System.Collections.Generic.List<Request>. This “list” is different than an F# list; it is a concrete, doubly-linked list. F# lists are immutable, recursive item/tail pairs, so F# views the former as a form of sequence (as it extends IEnumerable<T>). Thus, the Seq module calls in the return statement are the appropriate ones to use. They execute lazily, so filters should appear as early as possible; this reduces the number of latter transformations that may need to occur.

Looking at this example, if we were to sort first, the entire sequence would need to be sorted. Then, when we filter out the requests that are answered, we would remove items from that sequence. With sorting last, we only have to address the full sequence once, and we are sorting a (theoretically) smaller number of items. Conversely, we do have to run the map on the original sequence, as lastStatus is one of the calculated fields in the object created by ofRequestLite. Sometimes you can filter early, sometimes you cannot.

(Is this micro-optimizing? Maybe; but, in my experience, taking a few minutes to think through collection pipeline ordering is a lot easier than trying to figure out why (or where) one starts to bog down. Following good design principles isn't premature optimization, IMO.)

Getting a Database Connection

The example in the previous section has a final parameter of (db: LiteDatabase). As Giraffe sits atop ASP.NET Core, myPrayerJournal uses the traditional dependency injection (DI) container. Here is how it is configured:

/// Configure dependency injection
let services (bldr : WebApplicationBuilder) =
  // ...
  let db = new LiteDatabase (bldr.Configuration.GetConnectionString "db")
  Data.Startup.ensureDb db
  bldr.Services
    // ...
    .AddSingleton<LiteDatabase> db
  |> ignore
  // ...

The connection string comes from appsettings.json. Data.Startup.ensureDb makes sure that requests are indexed by user ID, as that is the parameter by which request lists are queried; this also registers the converter functions discussed above. LiteDB has an option to open the file for shared access or exclusive access; this implementation opens it for exclusive access, so we can register that connection as a singleton. (LiteDB handles concurrent queries itself.)

Getting the database instance out of DI is, again, a standard Giraffe technique:

/// Get the LiteDB database
let db (ctx : HttpContext) = ctx.GetService<LiteDatabase> ()

This can be called in any request handler; here is the handler that displays the journal cards:

// GET /components/journal-items
let journalItems : HttpHandler =
  requiresAuthentication Error.notAuthorized
  >=> fun next ctx -> backgroundTask {
    let  now   = now ctx
    let! jrnl  = Data.journalByUserId (userId ctx) (db ctx)
    let  shown = jrnl |> List.filter (fun it -> now > it.snoozedUntil && now > it.showAfter)
    return! renderComponent [ Views.Journal.journalItems now shown ] next ctx
    }

Making LiteDB Async

I found it curious that LiteDB's data access methods do not have async equivalents (ones that would return Task<T> instead of just T). My supposition is that this is a case of YAGNI. LiteDB maintains a log file, and makes writes to that first; then, when it's not busy, it synchronizes the log to the file it uses for its database. However, I wanted to control when that occurs, and the rest of the request/function pipelines are async, so I set about making async wrappers for the applicable function calls.

Here are the data retrieval functions:

/// Convert a sequence to a list asynchronously (used for LiteDB IO)
let toListAsync<'T> (q : 'T seq) =
  (q.ToList >> Task.FromResult) ()

/// Convert a sequence to a list asynchronously (used for LiteDB IO)
let firstAsync<'T> (q : 'T seq) =
  q.FirstOrDefault () |> Task.FromResult

/// Async wrapper around a request update
let doUpdate (db : LiteDatabase) (req : Request) =
  db.requests.Update req |> ignore
  Task.CompletedTask

And, for the log synchronization, an extension method on LiteDatabase:

/// Extensions on the LiteDatabase class
type LiteDatabase with
  // ...
  /// Async version of the checkpoint command (flushes log)
  member this.saveChanges () =
    this.Checkpoint ()
    Task.CompletedTask

None of these actually make the underlying library use async I/O; however, they do let the application's main thread yield until the I/O is done. Also, despite the saveChanges name, this is not required to save data into LiteDB; it is there once the insert or update is done (or, optionally, when the transaction is committed).

Final Thoughts

As I draft this, this paragraph is on line 280 of this post's source; the entire Data.fs file is 209 lines, including blank lines and comments. The above is a moderately long-winded explanation of what is nicely terse code. If I had used traditional C#-style POCOs, the code would likely have been shorter still. The backup of the LiteDB file is right at half the size of the equivalent RavenDB backup, so the POCO-to-BSON mapping paid off there. I'm quite pleased with the outcome of using LiteDB for this project.

Our final stop on the tour will wrap up with overall lessons learned on the project.

Categorized under , , ,
Tagged , , , , , , , , , , ,

Monday, November 29, 2021
  A Tour of myPrayerJournal v3: Bootstrap Integration

NOTE: This is the third post in a series; see the introduction for information on requirements and links to other posts in the series.

Many modern Single Page Application (SPA) frameworks include (or have plugins for) CSS transitions and effects. Combined with the speed of not having to do a full refresh, this is one of their best features. One might not think that a framework like htmx, which simply swaps out sections of the page, would have this; but if one were to think that, one would be wrong. Sadly, though, I did not utilize those aspects of htmx while I was migrating myPrayerJournal from v2 to v3; however, I will highlight the htmx way to do this in last section of this post.

myPrayerJournal v2 used a Vue plugin that provided Bootstrap v4 support; myPrayerJournal v3 uses Bootstrap v5. The main motivation I had to remain with Bootstrap was that I liked the actual appearance, and I know how it works. The majority of my “learning” on this project dealt with htmx; I did not want to add a UI redesign to the mix. Before we jump into the implementation, let me briefly explain the framework.

About Bootstrap

Bootstrap was originally called Twitter Bootstrap; it was the CSS framework that Twitter developed in their early iterations. It was, by far, the most popular framework at the time, and it was innovative in its grid layout system. Long before there was browser support for the styles that make layouts much easier to develop, and more responsive to differing screen sizes, Bootstrap's grid layout and size breakpoints made it easy to build a website that worked for desktop, tablet, or phone. Of course, there is a limit to what you can do with styling, so Bootstrap also has a JavaScript library that augments these styles, enabling the interactivity to which the modern web user is accustomed.

Version 5 of Bootstrap continues this tradition; however, it brings in even more utility classes, and supports Flex layouts as well. It is a mature library that continues to be maintained, and the project's philosophy seems to be “just enough” - it's not going to do everything for everyone, but in the majority of cases, it has exactly what the developer needs. It is not a bloated library that needs tree-shaking to avoid a ridiculous download size.

It is, by far, the largest payload in the initial page request:

  • Bootstrap - 48.6 kB (CSS is 24.8 kB; JavaScript is 23.8 kB, deferred until after render)
  • htmx - 11.8 kB
  • myPrayerJournal - 4.4 kB (CSS is 1.2 kB, JavaScript is 3.2 kB)

However, this gets the entire style and script, and allows us to use their layouts and interactive components. But, how do we get that interactivity from the server?

Hooking in to the htmx Request Pipeline

htmx provides several events to which an application can listen. In myPrayerJournal v3, I used htmx:afterOnLoad because I did not need the new content to be swapped in yet when the function fired. There are afterSwap and afterSettle events which will fire once those events have occurred, if you need to defer processing until those are complete.

There are two different Bootstrap script-driven components myPrayerJournal uses; let's take a look at toasts.

A Toast to Via htmx

Toasts are pop-up notifications that appear on the screen, usually for a short time, then fade out. In some cases, particularly if the toast is alerting the user to an error, it will stay on the screen until the user dismisses it, usually by clicking an “x” in the upper right-hand corner (even if the developer used a Mac!). Bootstrap provides a host of options for their toast component; for our uses, though, we will:

  • Place toasts in the bottom right-hand corner;
  • Allow multiple toasts to be visible at once;
  • Auto-hide success toasts; require others to be dismissed manually.

There are several different aspects that make this work.

The Toaster

Just like IRL toast comes out of a toaster, our toasts need a place from which to emerge. In the prior post, I mentioned that the footer does not get reloaded when a “page” request is made. There is also an element above the footer that also remains across these requests - defined here as the “toaster” (my term, not Bootstrap's).

/// Element used to display toasts
let toaster =
  div [ _ariaLive "polite"; _ariaAtomic "true"; _id "toastHost" ] [
    div [ _class "toast-container position-absolute p-3 bottom-0 end-0"; _id "toasts" ] []
    ]

This renders two empty divs with the appropriate style attributes; toasts placed in the #toasts div will display as we want them to.

Showing the Toast

Bootstrap provides data- attributes that can make toasts appear; however, since we are creating these in script, we need to use their JavaScript functions. The message coming from the server has the format TYPE|||The message. Let's look at the showToast function (the largest custom JavaScript function in the entire application):

const mpj = {
  // ...
  showToast (message) {
    const [level, msg] = message.split("|||")
  
    let header
    if (level !== "success") {
      const heading = typ => `<span class="me-auto"><strong>${typ.toUpperCase()}</strong></span>`
    
      header = document.createElement("div")
      header.className = "toast-header"
      header.innerHTML = heading(level === "warning" ? level : "error")
    
      const close = document.createElement("button")
      close.type = "button"
      close.className = "btn-close"
      close.setAttribute("data-bs-dismiss", "toast")
      close.setAttribute("aria-label", "Close")
      header.appendChild(close)
    }

    const body = document.createElement("div")
    body.className = "toast-body"
    body.innerText = msg
  
    const toastEl = document.createElement("div")
    toastEl.className = `toast bg-${level === "error" ? "danger" : level} text-white`
    toastEl.setAttribute("role", "alert")
    toastEl.setAttribute("aria-live", "assertlive")
    toastEl.setAttribute("aria-atomic", "true")
    toastEl.addEventListener("hidden.bs.toast", e => e.target.remove())
    if (header) toastEl.appendChild(header)
  
    toastEl.appendChild(body)
    document.getElementById("toasts").appendChild(toastEl)
    new bootstrap.Toast(toastEl, { autohide: level === "success" }).show()
  },
  // ...
}

Here's what's going on in the code above:

  • Line 4 splits the level from the message
  • Lines 6-20 (let header) create a header and close button if the message is not a success
  • Lines 22-24 (const body) create the body div with attributes Bootstrap's styling expects
  • Lines 26-30 (const toastEl) create the div that will contain the toast
  • Line 31 adds an event handler to remove the element from the DOM once the toast is hidden
  • Lines 32 and 34 add the optional header and mandatory body to the toast div
  • Line 35 adds the toast to the page (within the toasts inner div defined above)
  • Line 36 initializes the Bootstrap JavaScript component, auto-hiding on success, and shows the toast

(If you've never used JavaScript to create elements that are added to an HTML document, this probably looks weird and verbose; if you have, you look at it and think "well, they're not wrong…")

So, we have our toaster, we know how to put bread notifications in it - but how do we get the notifications from the server?

Receiving the Toast

The code to handle this is part of the htmx:afterOnLoad handler:

htmx.on("htmx:afterOnLoad", function (evt) {
  const hdrs = evt.detail.xhr.getAllResponseHeaders()
  // Show a message if there was one in the response
  if (hdrs.indexOf("x-toast") >= 0) {
    mpj.showToast(evt.detail.xhr.getResponseHeader("x-toast"))
  }
  // ...
})

This looks for a custom HTTP header of X-Toast (all headers are lowercase from that xhr call), and if it's found, we pass the value of that header to the function above. This check occurs after every htmx network request, so there is nothing special to configure; “page” requests are not the only requests capable of returning a toast notification.

There is one more part; how does the toast get to the browser?

Sending the Toast

The last paragraph gave it away; we set a header on the response. This seems straightforward, and is in most cases; but once again, POST-Redirect-GET (P-R-G) complicates things. Here are the final two lines of the successful path of the request update handler:

Messages.pushSuccess ctx "Prayer request updated successfully" nextUrl
return! seeOther nextUrl next ctx

If we set a message in the response header, then redirect (remember that XMLHttpRequest handles redirects silently), the header gets lost in the redirect. Here, Messages.pushSuccess places the success message (and return URL) in a dictionary, indexed by the user's ID. Within the function that renders every result (partial, “page”-like, or full results), this dictionary is checked for a message and URL, and if one exists, it includes it. (If it is returned to the function below, it has already been removed from the dictionary.)

/// Send a partial result if this is not a full page load (does not append no-cache headers)
let partialStatic (pageTitle : string) content : HttpHandler =
  fun next ctx -> backgroundTask {
    let  isPartial = ctx.Request.IsHtmx && not ctx.Request.IsHtmxRefresh
    let! pageCtx   = pageContext ctx pageTitle content
    let  view      = (match isPartial with true -> partial | false -> view) pageCtx
    return! 
      (next, ctx)
      ||> match user ctx with
          | Some u ->
              match Messages.pop u with
              | Some (msg, url) -> setHttpHeader "X-Toast" msg >=> withHxPush url >=> writeView view
              | None -> writeView view
          | None -> writeView view
    }

A quick overview of this function:

  • Line 4 determines if this an htmx boosted request (a “page”-like requests)
  • Line 5 creates a rendering context for the page
  • Line 6 renders the view to a string, calling partial or view with the page rendering context
  • Lines 10-13 are only executed if a user is logged on, and line 12 is the one that appends a message and a new URL

A quick note about line 12: the >=> operator joins Giraffe HttpHandlers together. An HttpHandler takes an HttpContext and the next function to be executed, and returns a Task<HttpContext option> (an asynchronous call that may or may not return a context). If there is no context returned, the chain stops; the function can also return an altered context. It is good practice for an HttpHandler to make a single change to the context; this keeps them simple, and allows them to be plugged in however the developer desires. Thus, the setHttpHeader call adds the X-Toast header, the withHxPush call adds the HX-Push header, and the writeView call sets the response body to the rendered view.

The new URL part does not actually make the browser do anything; it simply pushes the given URL onto the browser's history stack. Technically, the browser receives the content from the P-R-G as the response to its POST; as we're replacing the current page, though, we need to make sure the URL stays in sync.

Of note is that not all toasts are this complex. For example, the “cancel snooze” handler return looks like this:

return! (withSuccessMessage "Request unsnoozed" >=> Components.requestItem requestId) next ctx

...while the withSuccessMessage handler is:

/// Add a success message header to the response
let withSuccessMessage : string -> HttpHandler =
  sprintf "success|||%s" >> setHttpHeader "X-Toast"

No dictionary, no redirect, just a single response that will show a toast.

You made it - the toast section is toast! There is one more interesting interaction, though; that of the modal dialog.

Bootstrap's implementation of modal dialogs also uses JavaScript; however, for the purposes of the modals used in myPrayerJournal v3, we can use the data- attributes to show them. Here is the view for a modal dialog that allows the user to snooze a request (hiding it from the active list until the specified date); this is rendered a single time on the journal view page:

div [
  _id             "snoozeModal"
  _class          "modal fade"
  _tabindex       "-1"
  _ariaLabelledBy "snoozeModalLabel"
  _ariaHidden     "true"
  ] [
  div [ _class "modal-dialog modal-sm" ] [
    div [ _class "modal-content" ] [
      div [ _class "modal-header" ] [
        h5 [ _class "modal-title"; _id "snoozeModalLabel" ] [ str "Snooze Prayer Request" ]
        button [ _type "button"; _class "btn-close"; _data "bs-dismiss" "modal"; _ariaLabel "Close" ] []
        ]
      div [ _class "modal-body"; _id "snoozeBody" ] [ ]
      div [ _class "modal-footer" ] [
        button [ _type "button"; _id "snoozeDismiss"; _class "btn btn-secondary"; _data "bs-dismiss" "modal" ] [
          str "Close"
          ]
        ]
      ]
    ]
  ]

Notice that #snoozeBody is empty; we fill that when the user clicks the snooze icon:

button [
  _type     "button"
  _class    "btn btn-secondary"
  _title    "Snooze Request"
  _data     "bs-toggle" "modal"
  _data     "bs-target" "#snoozeModal"
  _hxGet    $"/components/request/{reqId}/snooze"
  _hxTarget "#snoozeBody"
  _hxSwap   HxSwap.InnerHtml
  ] [ icon "schedule" ]

This uses data-bs-toggle and data-bs-target, Bootstrap attributes, to show the modal. It also uses hx-get to load the snooze form for that particular request, with hx-target targeting the #snoozeBody div from the modal definition. Here is how that form is defined:

/// The snooze edit form
let snooze requestId =
  let today = System.DateTime.Today.ToString "yyyy-MM-dd"
  form [
    _hxPatch  $"/request/{RequestId.toString requestId}/snooze"
    _hxTarget "#journalItems"
    _hxSwap   HxSwap.OuterHtml
    ] [
    div [ _class "form-floating pb-3" ] [
      input [ _type "date"; _id "until"; _name "until"; _class "form-control"; _min today; _required ]
      label [ _for "until" ] [ str "Until" ]
      ]
    p [ _class "text-end mb-0" ] [ button [ _type "submit"; _class "btn btn-primary" ] [ str "Snooze" ] ]
    ]

Here, the form uses hx-patch to submit the data to the snooze endpoint. The target for the response, though, is #journalItems; this is the element that holds all of the prayer request cards. Snoozing a request will remove it from the active list, so the list needs to be refreshed; this will make that happen.

Look back at the modal definition; at the bottom, there is a “Close” button. We will use this to dismiss the modal once the update succeeds. In the Giraffe handler to snooze a request, here is its return statement:

return!
  (withSuccessMessage $"Request snoozed until {until.until}"
  >=> hideModal "snooze"
  >=> Components.journalItems) next ctx

Notice that hideModal handler?

/// Hide a modal window when the response is sent
let hideModal (name : string) : HttpHandler =
  setHttpHeader "X-Hide-Modal" name

Yes, it's another HTTP header! One can certainly get carried away with custom HTTP headers, but their very existence is to communicate with the client (browser) outside of the visible content of the page. Here, we're passing the name “snooze” to this header; in our htmx:afterOnLoad handler, we'll consume this header:

htmx.on("htmx:afterOnLoad", function (evt) {
  const hdrs = evt.detail.xhr.getAllResponseHeaders()
  // ...
  // Hide a modal window if requested
  if (hdrs.indexOf("x-hide-modal") >= 0) {
    document.getElementById(evt.detail.xhr.getResponseHeader("x-hide-modal") + "Dismiss").click()
  }
})

The “Close” button on our modal was given the id of snoozeDismiss; this mimics the user clicking the button, which Bootstrap's data- attributes handle from there. Of all the design choices and implementations I did in this conversion, this part strikes me as the most "hack"y. However, I did try to hook into the Bootstrap modal itself, and hide it via script; however, it didn't like initializing a modal a second time, and I could not get a reference to it from the htmx:afterOnLoad handler. Clicking the button works, though, even when it's done from script.

CSS Transitions in htmx

This post has already gotten much longer than I had planned, but I wanted to make sure I covered this.

  • When htmx requests are in flight, the framework makes it easy to show indicators.
  • I mentioned swapping and settling when discussing the events htmx exposes. The way this is done, CSS transitions will render as expected. They have a host of examples to spark your imagination.

As I was keeping the UI the same, I did not end up using these options; however, their presence demonstrates that htmx is a true batteries-included SPA framework.


Up next, we'll step away from the front end and dig into LiteDB.

Categorized under , , ,
Tagged , , , , , , , , , , , , , ,