Database, configuration and cleanup

August 16, 2016 Databases, sqlite, Web frameworks, Yesod No comments

We already have a database set up, some example data inserted and we’re using it in part of our application. Cool. Now it is time to discuss a little more about configuration of the database.
You might have spotted that we’re using runSqlPool instead of runSqlConn or even runSqlite – that is because – obviously – Yesod uses a pool of database connections. While it might not seem critical, it’s quite important for the configuration. First of all, pool size has to be defined. By default the scaffolding takes all startup parameters from YAML file – config/settings.yml (config/test-settings.yml for test configuration overrides). It’s loaded in Settings.hs, so if you ever need to add configuration options (and I guarantee that you will need to), do it there.

Database configuration is near the end of the file:

database:
  # See config/test-settings.yml for an override during tests
  database: "_env:SQLITE_DATABASE:civil-project.sqlite3"
  poolsize: "_env:SQLITE_POOLSIZE:10"

it has a quite convenient syntax, that allows deployment without modification of configuration files – if an environment variable with a given name (here: SQLITE_DATABASE and SQLITE_POOLSIZE) exists, it is used as the value, otherwise the default (civil-project.sqlite3 and 10) is taken. Now, while analysis of pool size have no sense here, since that’s a parameter used mostly for performance reasons, the default location of the database is quite unfortunate. It leaves us with stuff between consecutive application runs, require to manage the schema during development etc. Or manually delete database file each time, which is not that bad, but not perfect either, especially that it lies in the middle of source.

A perfect solution would be to run database in memory. This is possible, and not very hard – theoretically it should be sufficient to change configuration option to database: "_env:SQLITE_DATABASE:':memory'". Not obvious change, as it requires additional apostrophes, but not hard as well. And that’s enough to run SQLite database in memory. That would solve a lot of problems with the schema – database would be created from scratch each time. We’d have to keep example data somewhere in source, but we’ll need that for test reasons anyway so that’s no big deal. So, perfect, right?

No.

Unfortunately, there is one issue that causes this great plan to fail – in memory database does not play well with connection pools. Connections from pool are broken when there is no activity, and this causes in memory database to be deleted. At least that’s the scenario described on Yesod’s Google Groups.

So, there are two choices – we can either use a pool and a disk database or a single connection and in memory database. Since we’re now dealing with development issues, not testing, we can live with disk storage (we’ll rethink it when designing our tests). However, to provide some minimal encapsulation, we’re going to create a directory, workspace, and put the database there. Just create the directory and change database location (database: "_env:SQLITE_DATABASE:workspace/civil-project.sqlite3" is the line you might be looking for), remove the old database (or move it to workspace) and that’s it for the database location.

Now let’s do some cleanup in the code. First, routes. In our case, /projects redirects to ProjectsR handler, and /project/[x] to ProjectR handler. That’s not exactly how most web applications behave – in typical case it would be /project and /project/[x] (note lack of plural form in the first one). It’s not a big change, so let’s do it. Now our routes contain:

/static StaticR Static appStatic
/auth   AuthR   Auth   getAuth

/favicon.ico FaviconR GET
/robots.txt RobotsR GET

/ HomeR GET POST

/comments CommentR POST

/project ProjectsR GET
/project/#Int ProjectR GET

I’m not going to get rid of the scaffolding routes yet (/comments, /), but in the future we’re gonna remove them, no worries.
Second thing is a little more complicated – we’re going to show all projects from our database in the main list, not just the ones we have hardcoded (that’s no difference for now, but in the next post we’re gonna explore forms and add some new projects – it’ll make a difference then).
You can start with removing import Database.Project(projects) from Handler/Projects.hs (that’s the easiest part), and then do the magic.
Database actions run in their own monad, which cannot be easily mixed with the one from layout. Therefore the code will complicate quite a bit:

getProjectsR = do
    projects <- runDB $ do
       dbProjects <- selectList [] []
       return $ map entityVal dbProjects
    defaultLayout $ do
        setTitle "Projects"
        $(widgetFile "projects")

As you can see, there is one “master monad” wrapping two smaller ones – one for fetching existing projects from database, the other for generating HTML response. Check it out, it really works!
There are a few things worth noting here – first of all, this will work only for small lists of projects – they are loaded into memory all at once, thus big lists would cause out of memory error. Second is the entityVal function, which extracts value from a database record. You might remember that in Handler/Project.hs we achieved similar functionality by pattern matching to Entity projectId project (exact code was Entity _ project). Both are fine, feel free to use whichever one suits you best.

And that’s it – we’ve managed to use DB in project list as well! Now, since we are already able to read from database, it would be nice to be able to write something. And that is what we’ll be dealing with in the next post – forms and simple data upload.

Stay tuned!

Attaching database to Yesod app

August 15, 2016 Databases, sqlite, Web frameworks, Yesod No comments

Today we’re going to attach a real database to our application and stop using hardcoded list of projects (well, partially – we’ll still use it to populate database on startup). First we need to understand several things – where is the model stored, how to fetch the data, how to perform database migration.

For now we need just a simple feature of fetching project records by their unique identifier – not the one assigned by Persistent (although this would work out-of-the-box), but the <code>identifier</code> field. Luckily, Persistent offers a feature of assigning unique identifiers to fields (not supported in MongoDB) – fields starting with uppercase letter are assumed to create a new index. in our case – add UID identifier to Project definition in Domain/Project.hs. That’s it for the index, now let’s go to fetching – that’s also fairly easy, it’s sufficient to change selectPage implementation to:

selectPage :: Int -> Widget
selectPage projectId = do
  (Entity _ project) <- handlerToWidget $ runDB $ getBy404 $ UID projectId
  renderPage project

And getting is done. We also get 404: NotFound handling for free – if a record is not found in database, 404 is automatically returned and no further processing is needed. The Entity type here might be a little confusing – it means that the value is taken directly from database and that it is an actual database record. Its first field (here omitted) is id assigned automatically to the data (we don’t need it here). So, that’s it, right? Should it already work? Let’s try it out: stack exec -- yesod devel. It compiles, that’s a good sign. Let’s navigate to localhost:3000/project/1. Whoops, is it an Internal Server Error? What has gone wrong?

The error obviously says that project table doesn’t exist in the database. Oh, right. We just defined migration, but did not run it! Application compiled, because we aren’t obliged to run migration – for example, database may be owned by another team or may be used by more than one application – that’s a logic error, which is not typechecked right now in Yesod. Let’s run the migration then. The scaffolding runs migrations in makeFoundation function, in Application.hs file. The function is quite long, but the important part is runLoggingT (runSqlPool (runMigration migrateAll) pool) logFunc. It seems that some migration is already done, what’s going on?

That’s migration for data structures of the scaffolding – users, comments. Where are they defined? If you inspect Model.hs, you’ll find the following code:

share [mkPersist sqlSettings, mkMigrate "migrateAll"]
    $(persistFileWith lowerCaseSettings "config/models")

Apperently that’s also some Template Haskell magic – and entities are defined in config/models.
This file looks quite reasonable:

User
    ident Text
    password Text Maybe
    UniqueUser ident
    deriving Typeable
Email
    email Text
    userId UserId Maybe
    verkey Text Maybe
    UniqueEmail email
Comment json -- Adding "json" causes ToJSON and FromJSON instances to be derived.
    message Text
    userId UserId Maybe
    deriving Eq
    deriving Show

 -- By default this file is used in Model.hs (which is imported by Foundation.hs)

These are simply data definitions, structurally identical to the one we put into Domain/Project.hs. So, have we put our definitions in a wrong file? Not quite. Yesod as a framework is quite composable. We can use structues provided by scaffolding or not use them, it’s not going to blame us for either of these choices.
So theoretically we could just add our definition of Project here and be done with it. If you prefer this way – go ahead, I won’t blame you. I don’t like the idea, because it causes two separate domains to be mixed, and be tough later on. You can do two other things to solve this – either import Domain.Project everywhere where it is needed, or rely somehow on semi-default behavior. I’ve chosen to go for semi-default behavior – extract models to a separate file, but handled the same way the default one is. I’m not going to argue whether this is better or worse than keeping separate Domain.Project – both have their pros and cons. For example, by putting data definitions to separate file which is included in Import.hs, whole application may start to rely on these data structures. This may cause quite a lot of trouble during refactoring later on, even though we have a strong type system to save us from the simplest mistakes. On the other hand, importing Domain/Project.hs in every place is not what you would expect at first. For now I’m going to stay with the scaffolding’s approach, and if it proves hard to maintain – refactor.

Let’s create a config/projectModels file with the content from the quasiquoted part of Domain/Project.hs:

Project
      identifier Int
      name Text
      shortName Text
      deadline UTCTime
      UID identifier
      deriving Show

Now we can remove Domain/Project.hs and all it’s imports – it’ll soon become visible by default (from Import.hs). To make it visible in all (unfortunately) files that import Import.hs add the following call to Model.hs:

share [mkPersist sqlSettings, mkMigrate "migrateProject"]
    $(persistFileWith lowerCaseSettings "config/projectModels")

That’s all nice and fine, but we still don’t have our migration! To add it to application startup add runLoggingT (runSqlPool (runMigration migrateProject) pool) logFunc line to Application.hs, just below runLoggingT (runSqlPool (runMigration migrateAll) pool) logFunc. We could actually remove the latter, but it may be useful later on when learning more Yesod’ish patterns.

You might have guessed by now that mkMigrate string argument is name of function that will be created for migration. That’s quite a common pattern in places where Yesod uses Template Haskell.

In a big web application I would seriously consider either leaving out this pattern or importing everything to avoid code tangling – but I belive that this app will be small enough to ignore this problem – after all our main objective is to learn some Yesod here, isn’t it?

Now, that’s all nice and fine, but now we have 404s on each project read, and actually we’d prefer to read an actual record from the database, right? So we have to populate the database. Today we’ll do it in a brute-force way, but soon we’ll be honing this as well.

For now, it’s sufficient to add _ <- mapM (\p -> runLoggingT (runSqlPool (insertBy p) pool) logFunc) projects to makeFoundation, just after migrations. You might be tempted to use insert instead of insertBy, but this would cause your application to require much more maintenance – namely, removal of the database after each startup (SQLite would return an error when inserting entries with same UID, and error on startup means that whole application needs to be shut down). Of course, such solution is ugly and adds hacks to production code just to handle development needs – still, it’s ok for us now.

Try out the app after changes – functionality is same as before, but now data is fetched straight from database, which means that we can also insert new entries at runtime. We’re going to deal with forms and inserting in near future, but the next post will be again about databases – where is it? Why our list of projects is still a constant?

Stay tuned!

Using Persistent tables

August 10, 2016 Web frameworks, Yesod No comments

Databases are important. We need to keep all kinds of data in our applications – users, workspaces, messages, even things like selected UI theme. Yesod’s typical answer to this problem is Persistent. Persistent is a Haskell library for accessing databases – ranging from SQLite to MongoDB. It provides a type-safe access from Yesod to database and manages database schema and its changes (that’s actually done in runtime, but connection won’t be established without proper schema). Of course, not all migrations are possible to detect automatically – for example renaming a field will not be executed automatically. You can find more details about migrations in the Yesod Book, chapter about Persistent.

Luckily, our case is pretty straightforward – we want to store records for projects. For now we need only current state of the projects, so we’re going to use the usual facilities offered by Persistent. This involves some Template Haskell, but don’t worry, it’s quite straightforward. First we need to define our data type – but in a little special, Persistent-dedicated way. We’re gonna leave our pair-driven implementation and move to more complex data type. Our Project type will consist of a numeric identifier, textual name, short version of name and a deadline. I was quite tempted to leave out the deadline field since it introduces a bit of trouble, but after all – what’s a project without a deadline?

So, let’s create a Domain/Project.hs file with the following content:

module Domain.Project (Project(Project, projectIdentifier, projectName, projectShortName, projectDeadline)) where

import Import

share [mkPersist sqlSettings, mkMigrate "migrateProjects"] [persistLowerCase|
Project
  identifier Int
  name Text
  shortName Text
  deadline UTCTime
  deriving Show
|]

that’s quite dense, so probably requires some explaination. The part between | symbols is the actual data type definition. It is the input to the magic performed by Template Haskell, which results in generating field “accessors” exported in the first line: projectIdentifier, projectName, projectShortName, projectDeadline. You might guess that these are names of fields appended to name of data type and camelCased – there actually is a valid reason to do that – it is really helpful in avoiding name clashes (imagine field id). Actually, there is also a reason why the first field is called identifier and not simply id – Persistent internally creates an index field for each type, thus resulting in a clash. The type of this automatically generated id varies between backend databases – in most SQL-based ones it’s Int, but in MongoDB it’s String. Of course, both are nicely hidden under a type alias [[DatatypeName]]Id.

persistLowerCase is just a way of telling Persistent how to map field names to DB names, so let’s not focus on it. share is more interesting – it’s a helper function that takes a list of function an a quasiquotation as arguments (quasiquotes are these funny expressions strucured as [sth|sth else] – you can read more about them here) and passes the second one as argument for each of elements in the array. In our case these are mkPersist responsible for DB access and mkMigrate responsible for generating new DB schema.

And the last part that might be a little tricky – exports. module Domain.Project (Project(Project, projectIdentifier, projectName, projectShortName, projectDeadline)) – that’s just a way constructors and fields are exported in Haskell, TypeName(TypeConstructor, OtherTypeConstructor, field1, field2...).

Great, we’ve already done the tough job, now we have to clean up. First let’s change Database/Projects.hs. We’ve already prepared our application to interact with a database, but the interaction itself will be the subject of the next post. For now let’s just make it work. Here’s the new content:

module Database.Projects (projects) where
import Import

import Data.Time.Clock
import Domain.Project

time:: Integer -> Integer -> UTCTime
time a b = UTCTime (fromGregorian a 1 1) (secondsToDiffTime b)

projects::[Project]
projects = [Project 1 "Project 1" "P1" $ time 0 0,
  Project 3 "Substantial" "P2"  $ time 1 0,
  Project 5 "What the hell" "WTH"  $ time 100 0,
  Project 10 "The last project" "TLP"  $ time 10 10]

Only tricky thing here is the UTCTime creation. for some reason secondsToDiffTime is not imported by default, thus we need to explicitly add import Data.Time.Clock. This time function is also quite weird, but I guess we can live with that for now – after all we’ll get rid of such syntax in this file in another post (and if we won’t, we’ll refactor it soon).

Minor fixes are also needed in Handler/Project.hs (mostly substituting fst x with projectIdentifier x) and templates/projects.hamlet (same story + pattern matching in $forall). Handler/Projects.hs might also need a little change, namely importing Domain/Project.hs into the scope.

I’ve also removed copyright from the footer. We don’t need it now and it looked awful.

Quite some changes today! Remember that if something doesn’t work for you you can always check out the exact code that I’ve use during writing this post – it’s available at GitHub.

Next episode, in which we’re going to really attach a database to the application comes soon.

Stay tuned!

The foundation datatype

August 8, 2016 Web frameworks, Yesod No comments

This post will introduce some magic that you’ll use on a daily basis when using Yesod. This magic is mostly related to connection with the outside world – while Handler and Widget have IO in their type signatures, there are recommended ways of dealing with it – and this recommended way includes a foundation datatype.

Foundation datatype is a type that is accessible in all your Widget‘s and Handler‘s. In fact, it is even in the signature of your application – in Yesod App, App is your foundation datatype (that’s the name in the default scaffolding, you can change it if you wish to). You can keep anything there – default scaffolding puts there some server settings, connection pools, logger etc., but you can put virtually anything there – it’s just a typical data type.

This time we’re going to use it in a very simple way – to inject application name into the widgets and use it in the title – but on future posts we’ll be using the foundation value in much more sophisticated scenarios, such as database access or configuration. First let’s add the name to our data type – since this is just a tutorial, we’ll add it as a top-level field. So, in Foundation.hs change the definition of data App to

data App = App
    { appSettings    :: AppSettings
    , appStatic      :: Static -- ^ Settings for static file serving.
    , appConnPool    :: ConnectionPool -- ^ Database connection pool.
    , appHttpManager :: Manager
    , appLogger      :: Logger
    , appName        :: Text
    }

Next step, initialization. In Application.hs in code of makeFoundation function add appName <- return "Project M" line. That’s it, you already have your foundation data initialized. A little magic here is performed by the following line: let mkFoundation appConnPool = App {..} – the {..} syntax binds together names that exist in the scope and create a single data record. By the way, you can also use it the other way around: App {..} <- getYesod will bring all the fields to the current scope. I believe it generally clutters the scope and should be avoided. Still, sometimes it can come in handy. getYesod is a wrapped object of the foundation type, which is usually accessed by the syntax mentioned above (app <- getYesod) – yet, it’s a typical Yesod value, so there is no real reason (except maybe convenience) to stick to this method.

In the last step we’re going to change Handler/Project.hs and it’s rendering function. Let’s change the first two lines of renderPage function to:

app <- getYesod
setTitle $ (toHtml $ (appName app)) ++ ": " ++ projectTitle

And that’s it, title on your development server should already change. I’ll change the display to format appName: projectName(projectId), by changing the where clause from projectTitle = (toHtml projectId) ++ (toHtml projectName) to projectTitle = (toHtml projectName) ++ "(" ++ (toHtml projectId) ++ ")".

That’s it for today – soon we’re going to explore the way of using databases with Yesod, but first – in the next post – we’ll have to deal with our data type and make it more useful.

Stay tuned!

What’s a Widget, Precious?

August 2, 2016 Web frameworks, Yesod No comments

Widgets in Yesod are a way to add some nice features – like order independence – to created parts of page.  That’s because Widgets are more semantic elements than a set of tags – they also contain some information about tag location – for example if it should be inserted in the header or body of the page. For example, let’s assume we want to add a button that will redirect from each detailed project page to the list of all projects.

To do this, we’re gonna create a Hamlet template (in templates/back-to-projects.hamlet) for the widget first. It’ll contain the following code:

<div .comeback-link>
  <a .btn .btn-default href="@{ProjectsR}" role="button">Back to project list

Then let’s create a Cassius template (templates/back-to-projects.cassius) for styles:

.comeback-link
  margin-top: 10px
  margin-bottom: 10px

And in the end let’s use these templates in our Handler/Project.hs:

renderPage :: (Int, Text) -> Widget
renderPage projectEntry = do
  setTitle $ ("Project " ++ projectTitle)
  $(widgetFile "project")
  $(widgetFile "back-to-projects")
  where
    projectTitle = (toHtml projectId) ++ (toHtml projectName)
    projectName = snd projectEntry
    projectId = fst projectEntry

As a result, we get a nice button below the description of the project – this way we can add widget after widget to the page. Of course, we can also compose them to create bigger widgets. Now, that’s not something very interesting – after all we just added a piece of HTML below our previous HTML and it appeared below, so what’s the fun? Well, a bit of fun happens with the stylesheet and JS files (we don’t have a JS file here). Stylesheet got minified and added to the page stylesheet, returned in the header. If we had a JS file, it would also be included as a script (or file, depends on configuration) in the header. Of course, it may be that you actually want the JS to appear in the body – for this, Yesod offers toWidgetBody and toWidgetHead functions, which let you modify HTML head and body respectively. Widgets can be used not only for web pages – there are also functions – like toWidgetMedia that can help you create all other types of data, including print styling or API.

One more note – if you wonder what a Widget is, it is just an alias for WidgetT <<AppName>> IO (), where <<AppName>> can be defined in Application.hs (by default it’s MyApp). It is important to note that Widget is typed per-application. And we’ll discuss why exactly is it important next week, when discussing foundation datatype.

Stay tuned!

404: Not Found

August 1, 2016 Web frameworks, Yesod No comments

In the previous post we’ve managed to create a page with links which are typechecked. I believe that’s really cool, but we’ve missed one ultra-important feature: error 404: Not Found. Current implementation (available with step-by-step changes all the time at GitHub) returns a default page in this case – with <<Unknown>> project. That’s an ugly workaroud for the fact that we didn’t know how to return errors back then. Now we’re going to learn how to do it.

Let’s start with splitting getProjectR (in Project.hs) into three separate functions, like that:

getProjectR :: Int -> Handler Html
getProjectR projectId = defaultLayout $ selectPage projectId

selectPage :: Int -> Widget
selectPage projectId = renderPage project where
  project = maybe (-1, ""::Text) id projectEntry
  projectEntry = find (\x -> (fst x) == projectId) projects

renderPage :: (Int, Text) -> Widget
renderPage projectEntry = do
  setTitle $ ("Project " ++ projectName)
  $(widgetFile "project")
  where
    projectTitle = (toHtml projectId) ++ (toHtml projectName)
    projectName = snd projectEntry
    projectId = fst projectEntry

I’ve added type signatures to top-level functions, both to make GHC stop complaining and to introduce another important topic which we’ll discuss later in the series – Widgets. For now let’s simply assume that Widget is part of a web page.

Now, since selectPage is the only entity aware that something might go wrong, we should change it to return a 404 page in such case. It actually turns out to even remove some code:

selectPage :: Int -> Widget
selectPage projectId = maybe notFound renderPage projectEntry
  where
    projectEntry = find (\x -> (fst x) == projectId) projects

Check it out – it really does the job!
When I started writing this post I thought about implementing it the brute force way (fromJust). However, classy-prelude, a modified version of standard library which Yesod relies on doesn’t have a fromJust function. That turned out being actually quite nice, as I believe current implementation to be quite elegant, while the brute-force one would by, by definition, a brute-force one.

Next episode – more about Widgets – coming still this week. By the way, sorry about the delay in adding this post, I wanted to add it during the weekend but was unable to. I’ll do my best to avoid such situations in the future.

Typesafe routes

July 28, 2016 Web frameworks, Yesod No comments

Greetings and salutations!

Todays’ post will be about – you guessed it – typesafe routes in Yesod. First, let’s talk a little about what does a “typesafe route” actually mean. Type safety is related to compile time (perhaps with exception for dependent types, but let’s not dig into it for now), so all guarantees you can get are compile-time guarantees. Compile time guarantees are related to the structure of data rather than the data itself. So, for example, if we used a typesafe route for the example from previous post (/project/#Int) it would prevent us from directing to /project/phew, but not from /project/423, regardless whether /project/423 did actually exist – it is not possible to decide in compile time, and it might exist, so the route is allowed.

And there is one more drawback – typesafe routes are type-safe only in scope of your site (and of course do not guarantee reachability, network connection etc.). It might be a lot or a little, depending on your use case – still, it’s more than other frameworks offer out-of-the-box. What’s important is to remember that – while they might simplify your life – they are no magical way to solve dead link problems. And of course 404s will still occur, so usual testing is still needed.

Enough talking, let’s write some code. Today we’re going to make a link-per-project on the /projects route. We already have the routes both for main projects page and for each project (no content though!), so today we’re just going to link them together.

First, let’s extract the projects to a separate file – in the end it’ll be a database, but for now a separate module is sufficient. Let’s create a new top-level directory, Database, a module Projects there and export projects constant from there:

module Database.Projects (projects) where
import Import

projects::[(Int, Text)]
projects = [(1, "Project 1"), (2, "Project 2"), 
  (4, "My extra project"), (5, "Project 10")]::[(Int, Text)]

then add an import to both Projects.hs and Project.hs:
import Database.Projects(projects)
.

In Projects.hs remove projects = ["Project 1", "Project 2", "My extra project", "Project 10"]::[Text] (it is loaded from Database.Projects module) and in projects.hamlet change projectName variable assignment to a tuple: (projectId, projectName) (compiler will complain that projectId is not used, but we’ll use it in a minute). You can check that everything still works fine.

Now let’s change Project.hs file. We’re going to display both title and id of the project. Database.Projects is already loaded, but there is a little trick: it might be that requested project does not exist. Ideally we should return a 404: NotFound in such case, but for now we can live with some default behavior (no worries – we’re gonna raise a 404 in the next post)

getProjectR projectId = defaultLayout $ do
  setTitle $ ("Project " ++ (createTitle projectId))
  $(widgetFile "project")
  where
    createTitle projId = (toHtml projId) ++ projectName
    projectName = toHtml $ maybe "" snd projectEntry
    projectEntry = find (\x -> (fst x) == projectId) projects

this might look a little complicated, but the toughest thing here is handling the default case. Let’s also add 

Name: #{projectName}

to project.hamlet, so that it’ll be easier to see if it works. Check that everything works fine – it should.

And now, time for the clue – routes. They’re quite simple – just change

<li .list-group-item>#{projectName}

to

<li .list-group-item>
  <a href="@{ProjectR projectId}">#{projectName}</a>

in projects.hamlet.

And that’s it. Congratulations, you’ve just created a set of type safe routes in Yesod! Next episode – about handling failure cases – comes still this week.

Stay tuned!

Routes with parameters

July 25, 2016 Web frameworks, Yesod No comments

Hi all,

sorry for the long break. It started with me getting the last DLC for The Witcher 3 – Blood & Wine (check it out if you didn’t play it yet), and then loads of stuff happened. Anyway, I’m back and I’m going to do my best to make up for the delay. Let’s start with parametrized routes, like /project/10, eat/200 etc. We’re going to add support for such routes in our project management app for each of the projects, indexed by some unique ID (artificially given).

Yesod handles routes in a relatively simple way – whole route is splitted by ‘/’ (forward slash) and a list is generated – for example, /project/2 will translate to ["project", "2"]. Don’t worry about trailing slashes or double-slashes, Yesod has you covered – routes are translated to a canonical form and users are redirected to them. In Yesod you specify a route parameter by it’s expected type name preceeded by # (if the value is between two forward slashes) or * (in case of more complex type).

Hence adding a new route is simple – just add

/project/#Int ProjectR GET

line to config/routes and we’re done. Well, not entirely – we still need to add proper handler, and that’s what we’re going to do now.
To do it properly, create a Project.hs file and add it to all required places (you can check which places are required in one of earlier posts). It’s content should be simple:

module Handler.Project where
import Import

import Yesod.Bootstrap()

getProjectR :: Int -> Handler Html
getProjectR projectId = defaultLayout $ dodo
  setTitle $ ("Project " ++ (toHtml projectId))
$(widgetFile "project")

This also requires a projects.hamlet file, equally simple:

<div>
  <h1 .page-header> Project: #{show projectId}

aaaand it’s working – check it out yourself! Here’s a little bonus – on a request for a resource that doesn’t match the type signature (for example, project/x) Yesod will automatically return 404: Not Found without even bothering you. Current 404 page isn’t nice, but it’s customizeable, obviously (we’re gonna deal with it in the future).

You might’ve wondered what will happen if you create two routes matching the same pattern, for example /project/#Int and /project/#Text (overlapping part is everything that is parseable by /project/#Int). Try it out yourself! Apprently, quite a nice exception is thrown:

Overlapping routes:
("ProjectR","ProjectR")

but what if you actually want routes to overlap, and simply choose “numeric handler when argument is numeric, and textual when it isn’t”? Good news is that you don’t need to push parsing logic into the handler – you simply need to change the route definition from /project/#Int and /project/#Text to disable overlap checking – but only for these routes. If you do that, be aware – first matching route wins, so be careful with the order!

Next episode – linking with typesafe routes – coming still this week. Stay tuned!

Working on the Hamlet template

July 9, 2016 Web frameworks, Yesod No comments

If you’re following the series, you definitely saw how ugly our page was at the end of the last post. Well, good news – we’re gonna fix it right now. Before you close the window, let me explain: I’m not going to start doing graphical design here, as I do not know enough about it. Instead, I’m going to show you how to use Yesod with one of the best design libraries in the world – Bootstrap. Yesod has kind of built-in support for Bootstrap – you might’ve guessed it by the import Yesod.Form.Bootstrap3() line in Projects.hs. It’s actually not exactly what we’re looking for, so let’s change it to import Yesod.Bootstrap() and add Yesod.Bootstrap to build dependencies in .cabal file. You might need to run stack solver --update-config && stack build before next run.
And that’s it – Bootstrap is already included in our files. Now let’s get rid of the ugly projects.cassius and let’s replace it with some nice styling!

But before this, you might be a little surprised that including Bootstrap didn’t require us to add any additional CSS files. That’s not actually true – we simply already have one. It’s located in static/css/bootstrap.css. If you are able to access the site after running stack exec -- yesod devel, you have this file – otherwise the site wouldn’t even compile.

Now you can delete all the classes and IDs we used in previous post and replace the .hamlet file with something like this:

<div>
  <h1 .page-header> Available projects:
  <div .jumbotron>
  <ul .list-group>
    <li .list-group-item> Project 1
    <li .list-group-item> Project 2
    <li .list-group-item> Super project

looks much better now, doesn’t it?

The hardcoded project names are quite a shame, we should get rid of them. Let’s make our Hamlet template an actual template that is filled with some data! For the first shot let it be a simple list of project names – typed [Text]. Wonder why not [String]? Short answer is: it’s more efficient. For a slightly longer one you can check out The Yesod Book. Anyway, our goal now is to have a list of Texts passed to the template and same output rendered. To do this, we need to add a little big of logic to our template. Lines starting with $ are treated as special instructions, virtually typical Haskell operations. #{someName} is used for variable interpolation. Check out the code for projects.hamlet:

<div>
  <h1 .page-header> Available projects:
  <div .jumbotron>
    $if null projects
      There are no ongoing projects
    $else
    <ul .list-group>
      $forall projectName <- projects
      <li .list-group-item> #{projectName}

Quite a change, isn’t it? But it’s quite simple – if there are no elements in projects container we show a message, otherwise – list all elements one-by-one.

But the code doesn’t compile again. To make it work you have to add projects variable to the scope of widgetFile invocation. This can be done for example by adding

    where
      projects = ["Project 1", "Project 2", "My extra project", "Project 10"]::[Text]

to getProjectsR function.

And that’s it! we can feed the template via the list in Projects.hs file. Nice, eh?

Actually, no. Right now it’s even worse than it was – previously it was sufficient to modify projects.hamlet and yesod runner automatically reloaded all the stuff. Now the compilation has to be done manually, by running stack build. Have no fear, we will change this in the future. but the next step is different. The next step will introduce Yesod’s killer feature – typesafe routes.

Stay tuned!

To code or not to code – the Hamlet template language

June 21, 2016 Web frameworks, Yesod No comments

Since we managed to successfully create a route, next step is to create an actual page to show it to the user.
While you can use virtually any Haskell template engine with Yesod, the built-in one uses Hamlet, Cassius, Lucius and Julius as its input. They are used to generate HTML, CSS and JavaScript files from templates. These four languages are known as Shakespearean templates – Hamlet for generating HTML, Cassius and Lucius for CSS (we’ll focus on Cassius, as the only difference is syntax of a block) and Julius for JavaScript – before you get too excited, Julius is not a transpiler – you have to write actual JavaScript, it’ll just do some variable interpolation. If you’re looking for an actual Haskell-to-JavaScript transpiler, you may want to take a look at GHCJS or Haste.

Remember our handler for projects, located in Handler/Projects.hs? We’re going to add some actual content now. First let’s add loading the page to handler. Here’s the new code for it:

getProjectsR :: Handler Html
getProjectsR = defaultLayout $ do
	setTitle "Projects"
	$(widgetFile "projects")

it’s important that the new part ($(widgetFile "projects")) is the last invocation in series, because it uses results of the earlier ones. Here’s an important fact: this code will not compile yet. That’s actually good news, because it means that Yesod got us covered against a situation where we forget to add the content (or the template name is not valid). To feed the compilation process it is sufficient to add an empty file located at templates/projects.hamlet. But what fun is an empty file? Let’s fill it with some static content, for example this one:

<div>
  <h1 .customTitle> Available projects:
    <ul #projectList>
      <li> Project 1
      <li> Project 2
      <li> Super project

Hamlet’s syntax is fairly straightforward: basically it’s like HTML, but uses nesting instead of closing tags. Also, you should only use a single tag per line (using multiple is possible, but they are not interpreted – i.e., you get what you write. Just don’t do it) and start the line with tag. While this seems reasonable, it leads to problems with whitespaces – suddenly trailing whitespaces in line start being important. Hamlet’s workaround for this is to provide escape characters: lines ending with # have their trailing whitespaces preserved, and lines starting with \ have their starting whitespaces preserved. # is a pretty important character in Hamlet – it’s also used for variable interpolation (we’ll cover it in the next post of the series) and assigning IDs to DOM elements (see ul element in the example above). There are also several other useful features like support for Maybe or conditionals, but we don’t need them now, so let’s not bother ourselves with them. Instead, let’s add a little style to our list of projects.

To include a custom stylesheet for our projects route, it’s sufficient to create templates/projects.cassius file and Yesod will understand that we want to use it. Let’s fill it with some simple content for now:

#projectList
    text-align: center;
    margin-bottom: 30px

.customTitle
    color: #FF0000

div
    color: #00FF00

now it looks really bad. But obviously it worked and stylesheet was included – go to localhost:3000/projects to check it out (remember to run stack exec -- yesod devel before). What’s also worth noting, this stylesheet was included only on projects route (don’t believe me – check it out yourself at localhost:3000).

I can imagine some of you wheezing: “sure, that’s all nice and fine, but what’s the actual benefit from using a new language? We could just use plain CSS for this, and industry already has Sass and Less to make life easier”. Well, you’re right – I didn’t show anything you can’t easily do with CSS yet. Bad news is that I won’t do it today – variable interpolation comes next, and that’s quite a nice feature – although I believe it’s much more useful for HTML templates than CSS ones.

Let’s sum up for today – we’ve managed to create a part of page that is injected into default-layout, but can contain own CSS and JavaScript (and HTML code of course). We also found out that .hamlet file is the only required part of the page, and the server will not compile without it – CSS and JS files will be included only if available (but if they are available, they will be included automatically). We also created a really ugly piece of layout, which we’ll probably need to fix later on – but that’s not a big deal, since we used it only to prove that stylesheets will be added automatically. Up to now the new template languages weren’t really useful – they simply had a bit different syntax. In the next episode we’ll explore their actual power – variable interpolation.

Stay tuned!