NOTE: This is the final post in a series; see the introduction for information on requirements and links to other posts in the series.

We've gone in depth on several different aspects of this application and the technologies it uses. Now, let's zoom out and look at some big-picture lessons learned.

What I Liked

Generally speaking, I liked everything. That does not make for a very informative post, though, so here are a few things that worked really well.

Simplification Via htmx

One of the key concepts in a Representational State Transfer (REST) API is that of Hypermedia as the Engine of Application State (HATEOAS). In short, this means that the state of an application is held within the hypermedia that is exchanged between client and server; and, in practice, the server is responsible for altering that state. This is completely different from the JSON API / JavaScript framework model, even if they use GET, POST, PUT, and PATCH properly.

(This is a near over-simplification; the paper that initially proposed these concepts – in much, much more detail – earned its author a doctoral degree.)

The simplicity of this model is great; and, when I say “simplicity,” I am speaking of a lack of complexity, not a naïveté of approach. I was able to remove a large amount of complexity and synchronization from the client/server interactions between myPrayerJournal v2 and v3. State management used to be the most complex part of the application. Now, the most complex part is the HTML rendering; since that is what controls the state, though, this makes sense. I have 25 years of experience writing HTML, and even at its most complex, it simply is not.

LiteDB

This was a very simple application - and, despite its being open for any user with a Google or Microsoft account, I have been the only regular user of the application. LiteDB's setup was easy, implementation was easy, and it performs really well. I suspect this would be the case with many concurrent users. If the application were to grow, and I find that my suspicion was not borne out by reality, I could create a database file per user, and back up the data directory instead of a specific file. As with htmx, the lack of complexity makes the application easily maintainable.

What I Learned

Throughout this entire series of posts, most of the content would fall under this heading. There are a few things that did not fit into those posts, though.

htmx Support in .NET

I developed Giraffe.Htmx as a part of this effort, and mentioned that I became aware of htmx on an episode of .NET Rocks!. The project I developed is very F#-centric, and uses features of the language that are not exposed in C# or VB.NET. However, there are two packages that work with the standard ASP.NET Core paradigm. Htmx provides server-side support for the htmx request and response headers, similar to Giraffe.Htmx, and Htmx.TagHelpers contains tag helpers for use in Razor, similar to what Giraffe.ViewEngine.Htmx does for Giraffe View Engine. Both are written by Khalid Abuhakmeh, a developer advocate at JetBrains (which generously licensed their tools to this project, and produces the best developer font ever).

While I did not use these projects, I did look at the source, and they look good. Open source libraries go from good to great by people using them, then providing constructive feedback (and pull requests, if you are able).

Write about Your Code

Yes, I'm cheating a bit with this one, as it was one of the takeaways from the v1 tour, but it's still true. Writing about your code has several benefits:

• You understand your code more fully.
• Others can see not just the code you wrote, but understand the thought process behind it.
• Readers can provide you feedback. (This may not always seem helpful; regardless of its tone, though, thinking through whether the point of their critique is justified can help you learn.)

And, really, knowledge sharing is what makes the open-source ecosystem work. Closed / proprietary projects have their place, but if you do something interesting, write about it!

What Could Be Better

Dove-tailing from the previous section, writing can also help you think through your code; if you try to explain it, and and have trouble, that should serve as a warning that there are improvements to be had. These are the areas where this project has room to get better.

Deferred Features

There were 2 changes I had originally planned for myPrayerJournal v3 that did not get accomplished. One is a new “pray through the requests” view, with a distraction-free next-card-up presentation. The other is that updating requests sends them to the bottom of the list, even if they have not been marked as prayed; this will require calculating a separate “last prayed” date instead of using the “as of” date from the latest history entry.

The migration introduced a third deferred change. When v1/v2 ran in the browser, the dates and times were displayed in the user's local timezone. With the HTML being generated on the server, though, dates and times are now displayed in UTC. The purpose of the application is to focus the user's attention on their prayer requests, not to make them have to do timezone math in their head! htmx has an hx-headers attribute that specifies headers to pass along with the request; I plan to use a JavaScript call to set a header on the body tag when a full page loads (hx-headers is inherited), then use that timezone to adjust it back to the user's current timezone.

That LiteDB Mapping

I did a good bit of tap-dancing in the LiteDB data model and mapping descriptions, mildly defending the design decisions I had made there. The recurrence should be designed differently, and there should be individual type mappings rather than mapping the entire document. Yes, it worked for my purpose, and this project was more about Vue to htmx than ensuring a complete F#-to-LiteDB mapping of domain types. As I implement the features above, though, I believe I will end up fixing those issues as well.

Thank you for joining me on this tour; I hope it has been enjoyable, and maybe even educational.

NOTE: This is the third post in a series; see the introduction for information on requirements and links to other posts in the series.

Many modern Single Page Application (SPA) frameworks include (or have plugins for) CSS transitions and effects. Combined with the speed of not having to do a full refresh, this is one of their best features. One might not think that a framework like htmx, which simply swaps out sections of the page, would have this; but if one were to think that, one would be wrong. Sadly, though, I did not utilize those aspects of htmx while I was migrating myPrayerJournal from v2 to v3; however, I will highlight the htmx way to do this in last section of this post.

myPrayerJournal v2 used a Vue plugin that provided Bootstrap v4 support; myPrayerJournal v3 uses Bootstrap v5. The main motivation I had to remain with Bootstrap was that I liked the actual appearance, and I know how it works. The majority of my “learning” on this project dealt with htmx; I did not want to add a UI redesign to the mix. Before we jump into the implementation, let me briefly explain the framework.

About Bootstrap

Bootstrap was originally called Twitter Bootstrap; it was the CSS framework that Twitter developed in their early iterations. It was, by far, the most popular framework at the time, and it was innovative in its grid layout system. Long before there was browser support for the styles that make layouts much easier to develop, and more responsive to differing screen sizes, Bootstrap's grid layout and size breakpoints made it easy to build a website that worked for desktop, tablet, or phone. Of course, there is a limit to what you can do with styling, so Bootstrap also has a JavaScript library that augments these styles, enabling the interactivity to which the modern web user is accustomed.

Version 5 of Bootstrap continues this tradition; however, it brings in even more utility classes, and supports Flex layouts as well. It is a mature library that continues to be maintained, and the project's philosophy seems to be “just enough” - it's not going to do everything for everyone, but in the majority of cases, it has exactly what the developer needs. It is not a bloated library that needs tree-shaking to avoid a ridiculous download size.

It is, by far, the largest payload in the initial page request:

• Bootstrap - 48.6 kB (CSS is 24.8 kB; JavaScript is 23.8 kB, deferred until after render)
• htmx - 11.8 kB
• myPrayerJournal - 4.4 kB (CSS is 1.2 kB, JavaScript is 3.2 kB)

However, this gets the entire style and script, and allows us to use their layouts and interactive components. But, how do we get that interactivity from the server?

Hooking in to the htmx Request Pipeline

htmx provides several events to which an application can listen. In myPrayerJournal v3, I used htmx:afterOnLoad because I did not need the new content to be swapped in yet when the function fired. There are afterSwap and afterSettle events which will fire once those events have occurred, if you need to defer processing until those are complete.

There are two different Bootstrap script-driven components myPrayerJournal uses; let's take a look at toasts.

A Toast to Via htmx

Toasts are pop-up notifications that appear on the screen, usually for a short time, then fade out. In some cases, particularly if the toast is alerting the user to an error, it will stay on the screen until the user dismisses it, usually by clicking an “x” in the upper right-hand corner (even if the developer used a Mac!). Bootstrap provides a host of options for their toast component; for our uses, though, we will:

• Place toasts in the bottom right-hand corner;
• Allow multiple toasts to be visible at once;
• Auto-hide success toasts; require others to be dismissed manually.

There are several different aspects that make this work.

The Toaster

Just like IRL toast comes out of a toaster, our toasts need a place from which to emerge. In the prior post, I mentioned that the footer does not get reloaded when a “page” request is made. There is also an element above the footer that also remains across these requests - defined here as the “toaster” (my term, not Bootstrap's).

/// Element used to display toasts
let toaster =
  div [ _ariaLive "polite"; _ariaAtomic "true"; _id "toastHost" ] [
    div [ _class "toast-container position-absolute p-3 bottom-0 end-0"; _id "toasts" ] []
    ]

This renders two empty divs with the appropriate style attributes; toasts placed in the #toasts div will display as we want them to.

Showing the Toast

Bootstrap provides data- attributes that can make toasts appear; however, since we are creating these in script, we need to use their JavaScript functions. The message coming from the server has the format TYPE|||The message. Let's look at the showToast function (the largest custom JavaScript function in the entire application):

const mpj = {
  // ...
  showToast (message) {
    const [level, msg] = message.split("|||")
  
    let header
    if (level !== "success") {
      const heading = typ => `<span class="me-auto"><strong>${typ.toUpperCase()}</strong></span>`
    
      header = document.createElement("div")
      header.className = "toast-header"
      header.innerHTML = heading(level === "warning" ? level : "error")
    
      const close = document.createElement("button")
      close.type = "button"
      close.className = "btn-close"
      close.setAttribute("data-bs-dismiss", "toast")
      close.setAttribute("aria-label", "Close")
      header.appendChild(close)
    }

    const body = document.createElement("div")
    body.className = "toast-body"
    body.innerText = msg
  
    const toastEl = document.createElement("div")
    toastEl.className = `toast bg-${level === "error" ? "danger" : level} text-white`
    toastEl.setAttribute("role", "alert")
    toastEl.setAttribute("aria-live", "assertlive")
    toastEl.setAttribute("aria-atomic", "true")
    toastEl.addEventListener("hidden.bs.toast", e => e.target.remove())
    if (header) toastEl.appendChild(header)
  
    toastEl.appendChild(body)
    document.getElementById("toasts").appendChild(toastEl)
    new bootstrap.Toast(toastEl, { autohide: level === "success" }).show()
  },
  // ...
}

Here's what's going on in the code above:

• Line 4 splits the level from the message
• Lines 6-20 (let header) create a header and close button if the message is not a success
• Lines 22-24 (const body) create the body div with attributes Bootstrap's styling expects
• Lines 26-30 (const toastEl) create the div that will contain the toast
• Line 31 adds an event handler to remove the element from the DOM once the toast is hidden
• Lines 32 and 34 add the optional header and mandatory body to the toast div
• Line 35 adds the toast to the page (within the toasts inner div defined above)
• Line 36 initializes the Bootstrap JavaScript component, auto-hiding on success, and shows the toast

(If you've never used JavaScript to create elements that are added to an HTML document, this probably looks weird and verbose; if you have, you look at it and think "well, they're not wrong…")

So, we have our toaster, we know how to put bread notifications in it - but how do we get the notifications from the server?

Receiving the Toast

The code to handle this is part of the htmx:afterOnLoad handler:

htmx.on("htmx:afterOnLoad", function (evt) {
  const hdrs = evt.detail.xhr.getAllResponseHeaders()
  // Show a message if there was one in the response
  if (hdrs.indexOf("x-toast") >= 0) {
    mpj.showToast(evt.detail.xhr.getResponseHeader("x-toast"))
  }
  // ...
})

This looks for a custom HTTP header of X-Toast (all headers are lowercase from that xhr call), and if it's found, we pass the value of that header to the function above. This check occurs after every htmx network request, so there is nothing special to configure; “page” requests are not the only requests capable of returning a toast notification.

There is one more part; how does the toast get to the browser?

Sending the Toast

The last paragraph gave it away; we set a header on the response. This seems straightforward, and is in most cases; but once again, POST-Redirect-GET (P-R-G) complicates things. Here are the final two lines of the successful path of the request update handler:

Messages.pushSuccess ctx "Prayer request updated successfully" nextUrl
return! seeOther nextUrl next ctx

If we set a message in the response header, then redirect (remember that XMLHttpRequest handles redirects silently), the header gets lost in the redirect. Here, Messages.pushSuccess places the success message (and return URL) in a dictionary, indexed by the user's ID. Within the function that renders every result (partial, “page”-like, or full results), this dictionary is checked for a message and URL, and if one exists, it includes it. (If it is returned to the function below, it has already been removed from the dictionary.)

/// Send a partial result if this is not a full page load (does not append no-cache headers)
let partialStatic (pageTitle : string) content : HttpHandler =
  fun next ctx -> backgroundTask {
    let  isPartial = ctx.Request.IsHtmx && not ctx.Request.IsHtmxRefresh
    let! pageCtx   = pageContext ctx pageTitle content
    let  view      = (match isPartial with true -> partial | false -> view) pageCtx
    return! 
      (next, ctx)
      ||> match user ctx with
          | Some u ->
              match Messages.pop u with
              | Some (msg, url) -> setHttpHeader "X-Toast" msg >=> withHxPush url >=> writeView view
              | None -> writeView view
          | None -> writeView view
    }

A quick overview of this function:

• Line 4 determines if this an htmx boosted request (a “page”-like requests)
• Line 5 creates a rendering context for the page
• Line 6 renders the view to a string, calling partial or view with the page rendering context
• Lines 10-13 are only executed if a user is logged on, and line 12 is the one that appends a message and a new URL

A quick note about line 12: the >=> operator joins Giraffe HttpHandlers together. An HttpHandler takes an HttpContext and the next function to be executed, and returns a Task<HttpContext option> (an asynchronous call that may or may not return a context). If there is no context returned, the chain stops; the function can also return an altered context. It is good practice for an HttpHandler to make a single change to the context; this keeps them simple, and allows them to be plugged in however the developer desires. Thus, the setHttpHeader call adds the X-Toast header, the withHxPush call adds the HX-Push header, and the writeView call sets the response body to the rendered view.

The new URL part does not actually make the browser do anything; it simply pushes the given URL onto the browser's history stack. Technically, the browser receives the content from the P-R-G as the response to its POST; as we're replacing the current page, though, we need to make sure the URL stays in sync.

Of note is that not all toasts are this complex. For example, the “cancel snooze” handler return looks like this:

return! (withSuccessMessage "Request unsnoozed" >=> Components.requestItem requestId) next ctx

...while the withSuccessMessage handler is:

/// Add a success message header to the response
let withSuccessMessage : string -> HttpHandler =
  sprintf "success|||%s" >> setHttpHeader "X-Toast"

No dictionary, no redirect, just a single response that will show a toast.

You made it - the toast section is toast! There is one more interesting interaction, though; that of the modal dialog.

Modal Dialogs

Bootstrap's implementation of modal dialogs also uses JavaScript; however, for the purposes of the modals used in myPrayerJournal v3, we can use the data- attributes to show them. Here is the view for a modal dialog that allows the user to snooze a request (hiding it from the active list until the specified date); this is rendered a single time on the journal view page:

div [
  _id             "snoozeModal"
  _class          "modal fade"
  _tabindex       "-1"
  _ariaLabelledBy "snoozeModalLabel"
  _ariaHidden     "true"
  ] [
  div [ _class "modal-dialog modal-sm" ] [
    div [ _class "modal-content" ] [
      div [ _class "modal-header" ] [
        h5 [ _class "modal-title"; _id "snoozeModalLabel" ] [ str "Snooze Prayer Request" ]
        button [ _type "button"; _class "btn-close"; _data "bs-dismiss" "modal"; _ariaLabel "Close" ] []
        ]
      div [ _class "modal-body"; _id "snoozeBody" ] [ ]
      div [ _class "modal-footer" ] [
        button [ _type "button"; _id "snoozeDismiss"; _class "btn btn-secondary"; _data "bs-dismiss" "modal" ] [
          str "Close"
          ]
        ]
      ]
    ]
  ]

Notice that #snoozeBody is empty; we fill that when the user clicks the snooze icon:

button [
  _type     "button"
  _class    "btn btn-secondary"
  _title    "Snooze Request"
  _data     "bs-toggle" "modal"
  _data     "bs-target" "#snoozeModal"
  _hxGet    $"/components/request/{reqId}/snooze"
  _hxTarget "#snoozeBody"
  _hxSwap   HxSwap.InnerHtml
  ] [ icon "schedule" ]

This uses data-bs-toggle and data-bs-target, Bootstrap attributes, to show the modal. It also uses hx-get to load the snooze form for that particular request, with hx-target targeting the #snoozeBody div from the modal definition. Here is how that form is defined:

/// The snooze edit form
let snooze requestId =
  let today = System.DateTime.Today.ToString "yyyy-MM-dd"
  form [
    _hxPatch  $"/request/{RequestId.toString requestId}/snooze"
    _hxTarget "#journalItems"
    _hxSwap   HxSwap.OuterHtml
    ] [
    div [ _class "form-floating pb-3" ] [
      input [ _type "date"; _id "until"; _name "until"; _class "form-control"; _min today; _required ]
      label [ _for "until" ] [ str "Until" ]
      ]
    p [ _class "text-end mb-0" ] [ button [ _type "submit"; _class "btn btn-primary" ] [ str "Snooze" ] ]
    ]

Here, the form uses hx-patch to submit the data to the snooze endpoint. The target for the response, though, is #journalItems; this is the element that holds all of the prayer request cards. Snoozing a request will remove it from the active list, so the list needs to be refreshed; this will make that happen.

Look back at the modal definition; at the bottom, there is a “Close” button. We will use this to dismiss the modal once the update succeeds. In the Giraffe handler to snooze a request, here is its return statement:

return!
  (withSuccessMessage $"Request snoozed until {until.until}"
  >=> hideModal "snooze"
  >=> Components.journalItems) next ctx

Notice that hideModal handler?

/// Hide a modal window when the response is sent
let hideModal (name : string) : HttpHandler =
  setHttpHeader "X-Hide-Modal" name

Yes, it's another HTTP header! One can certainly get carried away with custom HTTP headers, but their very existence is to communicate with the client (browser) outside of the visible content of the page. Here, we're passing the name “snooze” to this header; in our htmx:afterOnLoad handler, we'll consume this header:

htmx.on("htmx:afterOnLoad", function (evt) {
  const hdrs = evt.detail.xhr.getAllResponseHeaders()
  // ...
  // Hide a modal window if requested
  if (hdrs.indexOf("x-hide-modal") >= 0) {
    document.getElementById(evt.detail.xhr.getResponseHeader("x-hide-modal") + "Dismiss").click()
  }
})

The “Close” button on our modal was given the id of snoozeDismiss; this mimics the user clicking the button, which Bootstrap's data- attributes handle from there. Of all the design choices and implementations I did in this conversion, this part strikes me as the most "hack"y. However, I did try to hook into the Bootstrap modal itself, and hide it via script; however, it didn't like initializing a modal a second time, and I could not get a reference to it from the htmx:afterOnLoad handler. Clicking the button works, though, even when it's done from script.

CSS Transitions in htmx

This post has already gotten much longer than I had planned, but I wanted to make sure I covered this.

• When htmx requests are in flight, the framework makes it easy to show indicators.
• I mentioned swapping and settling when discussing the events htmx exposes. The way this is done, CSS transitions will render as expected. They have a host of examples to spark your imagination.

As I was keeping the UI the same, I did not end up using these options; however, their presence demonstrates that htmx is a true batteries-included SPA framework.

Up next, we'll step away from the front end and dig into LiteDB.

NOTE: This is the second post in a series; see the introduction for information on requirements and links to other posts in the series.

If you are a seasoned Single Page Application (SPA) framework developer, you likely think about interactivity in a particular way. Initially, I focused on replacing each interactive piece in isolation. In the end, though, requests for “pages” returned almost everything but the HTML head info and the displayed footer - and I was happy about it. Keep that in mind as I walk you down the path I have already traveled; keep an open mind, and read to the end before forming strong opinions either way.

The $.05 Tour of Pug and Giraffe View Engine

Understanding the syntax of both Pug and Giraffe View Engine will help you if you click any of the source code example links. While a complete explanation of these two templating languages would make this long post much longer than it already is, here are some short examples of their syntax. Using a string variable who with the contents “World”, we will show both languages rendering:

<p id="example" class="greeting">Hello <strong>World</strong></p>

myPrayerJournal v2 used Pug templates in Vue to render the user interface. Pug uses indentation-sensitive tag/content pairs (or blocks), with JavaScript syntax for attributes, to generate HTML. To generate the example paragraph, the shortest template would look like:

p.greeting(id="example") Hello #[strong= who]

myPrayerJournal v3 uses Giraffe View Engine, which uses F# lists to generate HTML from a very HTML-looking domain-specific language (DSL). The example paragraph would be generated with:

p [ _id "example"; _class "greeting" ] [ str "Hello "; strong [] [ who ] ]

Given those examples, let's dig into the conversion.

The Menu

The menu across the top of the application was one of the first items I needed to convert. The menu needs to be different, depending on whether there is a user logged on or not. Also, if a user is logged on, the menu can still be different; the “Snoozed” menu item only appears if the user has any snoozed requests. The application uses Auth0 to manage users (which is how it is open to Microsoft and Google accounts), and I wanted to preserve this; my requests are tied to the ID provided by Auth0, so that did not need to change.

In the Vue version, the system used Auth0's SPA library that exposed whether there was a user logged on or not. Also, once a user was logged on, the API sent all the user's active requests, which included snoozed requests; once this API call returned, the application can turn on the “Snoozed” menu item. In the htmx version, though, this information is all generated on the server. My initial process was to use an hx-get to get the menu HTML snippet, using an hx-trigger of load to fill in this spot of the page when the page was loaded. I also (initially) implemented a custom HTML header to include in responses, and if that header was found, I would trigger a refresh on the menu; the eventual solution included the navbar in “page” refreshes.

(See the Vue “Navigation” component that became the Giraffe View Engine “navbar” function)

“New Page” in htmx

This leads directly into a discussion of how myPrayerJournal is still considered a SPA. In the Vue version, “pages” were Vue Single-File Components (SFCs) under the /components directory. (In the years since myPrayerJournal v1, the default Vue template has changed to place these SFCs under /views, while /components is reserved for shared components.) These view components rendered into a custom component within the main tag (using Vue router's router-view tag), while the nav component was reactive, based on the user logging on/off and snoozing requests.

In myPrayerJournal v3, “page” views target the #top section element. If the request is for a full page load, the HTML head content is rendered, as is the body's footer content; none of these change until a new version of the application is released. If the request is an htmx request, though, the only thing rendered is a new #top section, which includes the navigation bar and the page content. While this does approach a full “page load”, there are some key differences:

• The page contents are refreshed based on one HTTP request (no extra request or processing required for the navbar);
• The HTML head content is responsible for most of the large HTTP requests, such as those for JavaScript libraries (and is excluded from non-full-page views);
• The page footer is not included.

Note the difference between the full view layout and the partial layout. Also, within the application's request handlers, there is a partial return function that determines whether this is an htmx-initiated page view request (in which case a partial view is returned) or a full page request (which returns the entire template).

Updating the Page Title

One of the most unexpectedly-vexing parts of a SPA is determining how the browser's title bar will be updated when navigation occurs. (I understand why it's challenging; what I do not understand is why it took major frameworks so long to devise a built-in way of handling this.) Coming from that world, I had originally implemented yet another custom header, pushing the title from the server, and used a request listener to update the title if the header was present. As I dug in further, though, I learned that htmx will update the document title any time a request payload has an HTML title element in its head. If you look at both layouts in the preceding paragraph, you'll notice that they include a head element with a title tag. This is how easy it should be, and with htmx, this is how easy it is.

At this point, there is a pattern emerging. The thought process behind an htmx-powered website is much different than a JavaScript-based SPA framework; and, in the majority of cases, it has been less complex. Now, let me contradict what I just said.

POST-Redirect-GET

In myPrayerJournal v2, updating a prayer request followed this flow:

• Display the edit page, with the request details filled in
• When the user saved the request, return an empty 200 OK response
• Using Vue, display a notification, refresh the journal, then re-render the page where the user had been when they clicked “Edit” (there are multiple places from which requests can be edited)

While there are no redirects here, this is the classic traditional-web-application scenario where the “POST-Redirect-GET” (P-R-G) pattern is used. By using this pattern, the “Back” button on the browser still works. If you try to go back to the result of a POST request, the browser will warn you that your action will result in the data being resubmitted (not usually what you want to do). By redirecting, though, the result of a POST becomes a GET, which does not change any data. For traditional web applications, this is the user-friendliest way to handle updates.

In the htmx examples, they show an example of inline editing. This led to my first plan - change the request edit “page” to be a component, where the HTML for the displayed list was replaced by the form, and then the “Save” action returns the new HTML. This requires no P-R-G, as these actions have no effect on the “Back” button. It worked fine, but there were some things that weren't quite right:

• New requests needed their own page; I was going to have to duplicate the edit form for the “new” page, and introduce some complexity in determining how to render the results.
• Some updates required refreshing the list of requests, not just replacing the text and action buttons.

At this point, I was also starting to realize “if you think something is hard to do in htmx, you probably aren't trying to do it correctly.” So, I decided to try to replicate the “edit page” flow of v2 in v3. Creating the page was easy enough, and I was able to use the returnTo parameter in the function to both provide a “Cancel” button and redirect the user to the right place after saving. Easy, right? Well… Not quite.

htmx uses XMLHttpRequest (XHR) to send its requests, which has some interesting behavior; it follows redirects! When I submitted my form, it received the request (with htmx's HX-Request header set), and the server returned the redirect. XHR saw this, and followed it; however, it used the same method. (It was POSTing to the new URL.) The fix for this, though, was not easy to find, but easy to implement; use HTTP response code 303 (see other) instead of 307 (moved temporarily). Using this, combined with using hx-target="#top" on the form, allowed the P-R-G pattern to work successfully without double-POSTing and without a full-page refresh.

htmx Support in Giraffe and Giraffe View Engine

As I developed this, I was also building up extensions for Giraffe to handle the htmx request and response headers, as well as the attributes needed to generate htmx-aware markup from Giraffe View Engine. This project, called Giraffe.Htmx, is now published on NuGet. There are two packages; Giraffe.Htmx provides server-side support for the request and response headers, and Giraffe.ViewEngine.Htmx provides support for generating htmx attributes. I wrote about it when it was released, so I won't rehash the entire thing here.

Final UI Thoughts

htmx is much less complex than any other front-end JavaScript SPA framework I have ever used - which, for context, includes Angular, Vue, React, Ember, Aurelia, and Elm. Both in development and in production use, I cannot tell that the payloads are slightly larger; navigation is fast and smooth. Though I have yet to change anything since going live with myPrayerJournal v3, I know that maintenance will be quite straightforward (to be further explored in the conclusion post).

The UI for myPrayerJournal uses Bootstrap, a UI framework which has its own script, and htmx plays quite nicely with it. The next post in this series will describe how I interact with both Bootstrap and htmx, using modals and toasts on this “traditional” web application.

NOTES:

• This is post 6 in a series; see the introduction for all of them, and the requirements for which this software was built.
• Links that start with the text “mpj:” are links to the 1.0.0 tag (1.0 release) of myPrayerJournal, unless otherwise noted.

Up to this point in our tour, we've talked about data a good bit, but it has all been in the context of whatever else we were discussing. Let's dig into the data structure a bit, to see how our information is persisted and retrieved.

Conceptual Design

The initial thought was to create a document store with one document type, the request. The request would have an ID, the ID of the user who created it, and an array of updates/records. Through the initial phases of development, our preferred document database (RethinkDB) was going through a tough period, with their company shutting down; thankfully, they're now part of the Linux Foundation, so they're still around. RethinkDB supports calculated fields in documents, so the plan was to have a few of those to keep us from having to retrieve or search through the array of updates.

We also considered a similar design using PostgreSQL's native JSON support. While it does not natively support calculated fields, a creative set of indexes could also suffice. As we thought it through a little more, though, this seemed to be over-engineering; this isn't unstructured data, and PostgreSQL handles max-length character fields very well. (This is supposed to be a “minimalist” application, right?) A relational structure would fit our needs quite nicely.

The starting design, then, used 2 tables. request had an ID and a user ID; history had the request ID, an “as of” date, a status (created, updated, etc.), and the optional text associated with that update. Early in development, the journal view brought together the request/user IDs along with the latest history entry that affected the text of the request, as well as the last date/time an action had occurred on the request. When the notes capability was added, it got its own note table; its structure was similar to the history table, but with non-optional text and without a status. As snoozing and recurrence capabilities were added, those fields were added to the request table (and the journal view).

The final design uses 3 tables, 2 of which have a one-to-many relationship with the third; and 1 view, which provides the calculated fields we had originally planned for RethinkDB to calculate.

Database Changes (Migrations)

As we ended up using 3 different server environments over the course of this project, we ended up writing a DbContext class based on our existing structure. For the Node.js backend, we created a DDL file (mpj:ddl.js, v0.8.4+) that checked for the existence of each table and view, and also had the SQL to execute if the check failed. For the Go version (mpj:data.go, v0.9.6+), the EnsureDB function does a similar thing; looking at line 347, it is checking for a specific column in the request table, and running the ALTER TABLE statement to add it if it isn't there.

The only change that was required since the F#/Giraffe backend has been in place was the one to support request recurrence. Since we did not end up with a scaffolded EF Core initial migration/model, we simply wrote a SQL script to accomplish these changes (mpj:sql directory).¹

The EF Core Model

EF Core uses the familiar DbContext class from prior versions of Entity Framework. myPrayerJournal does take advantage of a feature that just arrived in EF Core 2.1, though - the DbQuery type. DbSets are collections of entities that generally map to an underlying database table. They can be mapped to views, but unless it's an updateable view, updating those entities results in a runtime error; plus, since they can't be updated, there's no need for the change tracking mechanism to care about the entities returned. DbQuery addresses both these concerns, providing lightweight read-only access to data from views.

The DbContext class is defined in Data.fs (mpj:Data.fs), starting in line 189. It's relatively straightforward, though if you have only ever seen a C# model, it's a bit different. The combination of val mutable x : [type] and the [<DefaultValue>] attribute are the F# equivalent of C#'s [type] x; declaration, which creates a variable and initializes reference types to null. The EF Core runtime provides these instances to their setters (lines 203, 206, 209, and 212), and the application code uses them via the getters (a line earlier, each).

The OnModelCreating overridden method (line 214) is called when the runtime first creates its instance of the data model. Within this method, we call the .configureEF function of each of our database types. The name of this function isn't prescribed, and we could define the entire model without even referencing the data types of our entities; however, this technique gives us a “configure where it's defined” paradigm with each entity type. While the EF “Code First” model creates tables that don't need a lot of configuring, we must provide more information about the layout of the database tables since we're writing a DbContext to target an existing database.

Let's start out by taking a look at History.configureEF (line 50). Line 53 says that we're going to the table history. This seems to be a no-brainer, but EF Core would (by convention) be expecting a History table; since PostgreSQL uses a different syntax for case-sensitive names, these queries would look like SELECT ... FROM "History" ..., resulting in a nice “relation does not exist” error. Line 54 defines our compound key (requestId and asOf). Lines 55-57 define certain properties of the entity as required; if we try to store an entity where these fields are not set, the runtime will raise an exception before even trying to take it to the database. (F#'s non-nullability makes this a non-issue, but it still needs to be defined to match the database.) Line 58 may seem to do nothing, but what it does is make the text property immediately visible to the model builder; then, we can define an OptionConverter<string>² for it, which will translate between null and string option (None = null, Some [x] = [x]). (Lines 60-61 are left over from when I was trying to figure out why line 62 was raising an exception, leading to the addition of line 58; they could safely be removed, and will be for a post-1.0 release.)

History is the most complex configuration, but let's take a peek at Request.configureEF (line 126) to see one more interesting technique. Lines 107-110 define the history and notes collections on the Request type; lines 138-145 define the one-to-many relationship (without a foreign key entity in the child types). Note the casts to IEnumerable<x> (lines 138 and 142) and obj (lines 140 and 144); while F# is good about inferring types in a lot of cases, these functions are two places it is not. We can use the :> operator for the cast, because these types are part of the inheritance chain. (The :?> operator is used for potentially unsafe casts.)

Finally, the attributes above each record type need a bit of explanation; each one has [<CLIMutable; NoComparison; NoEquality>]. The CLIMutable attribute creates a no-argument constructor for the record type, which the runtime can use to create instances of the type. (The side effect is that we may get null instances of what is expected to be a non-null type, but we'll look at dealing with that a bit later.) The NoComparison and NoEquality attributes keep F# from creating field-level equality and comparison methods on the types. While these are normally helpful, there is an edge case where they can raise NullReferenceExceptions, especially when used on null instances. As these record types are simply our data transfer objects (both from SQL and to JSON), we don't need the functionality anyway.

Reading and Writing Data

EF Core uses the “unit of work” pattern with its DbContext class. Each instance maintains knowledge of the entities it's loaded, and does change tracking against those entities, so it knows what commands to issue when .SaveChanges() (or .SaveChangesAsync()) is called. It doesn't do this for free, though, and while EF Core does this much more efficiently than Entity Framework proper, F# record types do not support mutation; if req is a Request instance, for example, { req with showAfter = 123456789L } returns a new Request instance.

This is the problem whose solution is enabled by lines 227-233 in Data.fs. We can manually register an instance of an entity as either added or modified, and when we call .SaveChanges(), the runtime will generate the SQL to update the data store accordingly. This also allows us to use .AsNoTracking() in our queries (lines 250, 258, 265, and 275), which means that the resultant entities will not be registered with the change tracker, saving that overhead. Notice that we don't specify that on line 243; since Journal is defined as a DbQuery instead of a DbSet, we get change-tracking-avoidance for free.

Generally speaking, the preferred method of writing queries against a DbContext instance is to define extension methods against it. These are static by default, and they enable the context to be as lightweight as possible, while extending it when necessary. However, since this context is so small, we've created 6 methods on the context that we use to obtain data.

If you've been reading along with the tour, we have already seen a few API handler functions (mpj:Handlers.fs) that use the data context. Line 137 has the handler for /api/journal, the endpoint to retrieve a user's active requests. It uses .JournalByUserId(), defined in Data.fs line 242, whose signature is string -> JournalRequest seq. (The latter is an F# alias for IEnumerable<JournalRequest>.) Back in the handler, we use db ctx to get the context (more on that below), then call the method; we're piping the output of userId ctx into it, so it gets its lone parameter from the pipe, then its output is piped to the asJson function we discussed as part of the API.

Line 192, the handler for /api/request/[id]/history, demonstrates both inserting and updating data. We attempt to retrieve the request by its ID and the user ID; if that fails, we return a 404. If it succeeds, though, we add a history entry (lines 201-207), and optionally update the showAfter field of the request based on its recurrence. Finally, the call on line 212 commits the changes for this particular instance. Since the .SaveChanges[Async]() methods return the number of records affected, we cannot use the do! operator for this; F# makes you explicitly ignore values you aren't either returning or assigning to a name. However, defining _ as a parameter or name demonstrates that we realize there is a value to be had, we just are not going to do anything with it.

We mentioned that CLIMutable record types could be null. Since record types cannot normally be null, we cannot code something like match [var] with null -> ...; it's a compiler syntax error. What we can do, though, is use the box operator. box “boxes” whatever value we have into an object container, where we can then check it against null. The function toOption in Data.fs on line 11 does this work for us; throughout the retrieval methods, we use it to return options for items that are either present or absent. This is why we could do the match statement in the /api/request/[id]/history handler against Some and None values.

Getting a DbContext

Since Giraffe sits atop ASP.NET Core, we use the same technique; we use the .AddDbContext() extension method on the IServiceCollection interface, and assign it when we set up the dependency injection container. In our case, it's in Program.fs (mpj:Program.fs) line 50, where we also direct it to use a PostgreSQL connection defined by the connection string “mpj”. (This comes from the unified configuration built from appsettings.json and appsettings.[Environment].json.) If we look back at Handlers.fs, lines 45-47, we see the definition of the db ctx call we used earlier. We're using the Giraffe-provided GetService<'T>() extension method to return this instance.

Our tour is nearing its end, but we still have a few stops to go. Next time, we'll look at how we generated documentation to tell people how to use this app.

¹ Writing this post has shown me that I need to either create a SQL creation script for the repo, or create an EF Core initial migration/model, so the database ever has to be recreated from scratch. It's good to write about things after you do them!

² This is also a package I wrote; it's available on NuGet, and I also wrote a post about what it does.