Postgres 11 has a nifty feature around partitions. When a partition exists for a range of values, when you insert into the parent table it’ll get routed to the correct partition. When you update that record’s partition key it will get moved to the correct partition. A default partition feature exists as well so that if you do try to insert something that doesn’t belong in any available partitions, it will be put there instead.

I wanted to use this feature to track transactions in a game economy. Where all transactions were stored with their transacted date as the partition key.

Here is the Wallet model.

defmodule Game.Wallets.Wallet do
alias Game.Wallets.Transaction
alias Game.Wallets.Wallet

use Ecto.Schema

import Ecto.Changeset

@type t :: %__MODULE__{
id: Ecto.UUID.t(),
name: String.t(),
balance: Decimal.t(),
locked_at: DateTime.t(),
deleted_at: DateTime.t(),
updated_at: DateTime.t(),
inserted_at: DateTime.t()
}
@primary_key {:id, :binary_id, autogenerate: false, read_after_writes: true}
@foreign_key_type :binary_id
schema("wallets") do
field(:name, :string)

field(:balance, :decimal, default: 0.0)

field(:locked_at, :utc_datetime)
field(:deleted_at, :utc_datetime)
field(:updated_at, :utc_datetime)
field(:inserted_at, :utc_datetime)

has_many(:transactions, Transaction)
end
end


Here is the Transaction model.

defmodule Game.Wallets.Transaction do
alias Game.Wallets.Wallet

use Ecto.Schema

import Ecto.Changeset

@type t :: %__MODULE__{
id: Ecto.UUID.t(),
occurred_at: DateTime.t(),
credit: Decimal.t(),
debit: Decimal.t(),
description: String.t(),
wallet_id: Ecto.UUID.t()
}
@primary_key {:id, :binary_id, autogenerate: false, read_after_writes: true}
@foreign_key_type :binary_id
schema("transactions") do
field(:occurred_at, :utc_datetime)

field(:credit, :decimal, default: 0.0)
field(:debit, :decimal, default: 0.0)

field(:description, :string)

belongs_to(:wallet, Wallet, type: :binary_id)
end
end


A fairly simple model, where I expect there to be millions of transactions. Something that can not live well on a single partition and be performant. So we need to make a migration that can handle this.

defmodule Game.Repo.Migrations.CreateWalletsAndTransactions do
use Ecto.Migration

def up do
execute """
CREATE TABLE wallets (
id          UUID NOT NULL DEFAULT uuid_generate_v4(),
name        TEXT NOT NULL,
balance     DECIMAL(20, 2) NOT NULL DEFAULT 0.0,
inserted_at TIMESTAMP NOT NULL DEFAULT now(),
updated_at  TIMESTAMP NOT NULL DEFAULT now(),
locked_at   TIMESTAMP,
deleted_at  TIMESTAMP,
PRIMARY KEY(id),
CONSTRAINT wallets_balance_ck CHECK(balance >= 0)
)
"""
end

def down do
end
end


We need to create the base paritition and default partition.

execute """
CREATE TABLE transactions (
id          UUID           NOT NULL DEFAULT uuid_generate_v4(),
wallet_id   UUID           NOT NULL REFERENCES wallets(id),
occurred_at TIMESTAMP      NOT NULL DEFAULT now(),
credit      DECIMAL(20, 2) NOT NULL DEFAULT 0.0,
debit       DECIMAL(20, 2) NOT NULL DEFAULT 0.0,
description TEXT
) PARTITION BY RANGE (occurred_at)
"""


Then we need to create the default partition.

execute "CREATE TABLE transactions_default PARTITION OF transactions DEFAULT"


Now the fun part is I needed a bunch of partitions created but didn’t want to type them all out by hand / copy paste.

start_date = beginning_of_month(~D[2019-04-01])

for months <- 0..47 do
create_partition("transactions", calculate_next_month(start_date, months))
end


The #create_partition/2, #beginning_of_month/1, and #calculate_next_month/2 are defined as follows.

defp create_partition(table, date) do
start_date = date

stop_date =
date

month =
start_date.month
|> Integer.to_string()

execute """
CREATE TABLE #{table}_p#{start_date.year}_#{month}
PARTITION OF #{table} FOR VALUES
FROM ('#{start_date}')
TO ('#{stop_date}')
"""
end

defp beginning_of_month(date) do
if date.day == 1 do
date
else
end
end

defp calculate_next_month(date, 0), do: date

defp calculate_next_month(date, months) do
calculate_next_month(next, months - 1)
end


I defined these methods in the Game.Repo.Migrations.CreateWalletsAndTransactions but definitely will extract these into a utility function to be used later.

Feel free to use it or manipulate it how ever. If you come up with a better solution I’d really like to see it.

Service objects are a handy tool to use in any ruby application, that has complex logic that needs to be extracted out of a controller or model. There are some nice benefits to extracting complex things into a more testable interface.

To avoid providing a contrived example, I will use similar code to how uploads on rubyfm are handled.

class UploadService

def initialize(user, options = {})
@user   = user
@logger = options[:logger] || NullObject.new
@errors = {}
end

def start(params)

logger.error { 'failed to save the upload' }
# more logging info on why
return false
end

unless transcode
logger.error { 'failed to transcode upload' }
return false
end

true
end

def transcode
end
end


And the UploadsController that utilizes the UploadService with the following.

class UploadsController < AuthorizedController
def create

authorize!(episode, :update?)
end

else
redirect_to(new_channel_episode_url(primary_channel))
end
end
end


Attempting to shove all of that logic into a controller action would simply be unmantainable and probably not well tested. With this setup you can isolate the service object from the request / controller tests and really excercise it at all of the potential fail points.

For example, we use Amazon to handle transcoding and I needed a test to ensure all issues would be caught and wrapped appropriately.

RSpec.describe UploadService, '#start', type: :service do
let(:user)    { Fabricate(:user) }
let(:service) { described_class.new(user) }

context 'when amazon fails to transcode' do
it 'returns false' do

expect(service.start(episode_id: SecureRandom.uuid)).to eq(false)
end
end

context 'when amazon transcode is successful' do
it 'returns true' do

expect(service.start(episode_id: SecureRandom.uuid)).to eq(true)
end
end
end


Go forth and use services where complex controller actions exist.

I’ve been writing a lot of C lately for a game I am working on. I am not a perfect programmer and I would like to catch my bugs before they arise. Thus, I am attempting to learn Rust in my free time.

## Backtrace

Printing a backtrace in C is not incredibly difficult to accomplish. Although the following is fairly primitive, it will aid in your ability to discern what is happening in your program

/// @file core/backtrace.c
#include <execinfo.h>
#include "backtrace.h"

/// @brief Prints a backtrace of up to the last 256 calls
///
/// @param [out] fd The file descriptor ID to write to
void print_backtrace(int fd) {
void* array[256];
size_t size = backtrace(array, 256);
backtrace_symbols_fd(array, size, fd);
}


The output will look like the following

bin/example(print_backtrace+0x22)[0x403002]
bin/example(test_block_insertion+0x218)[0x402b68]
bin/example(main+0x2f)[0x402eef]
/usr/lib/libc.so.6(__libc_start_main+0xf1)[0x7f3e3cb5c291]
bin/example(_start+0x2a)[0x4027fa]


As you can see, in a method test_block_insertion I call print_backtrace. This is incredibly helpful if you need to determine how deep in the program your issue occurred.

## Panic

I’ve been tinkering with Rust in my free time to get a better understanding of it and I have come to enjoy the panic! macro it employs. I like the verb and decided that it would work perfectly in my daily use.

With some alterations, I just wanted panic to do exactly what its name suggests, I want the program to panic with a message and abort.

/// @file core/panic.h
#pragma once

#include <stdlib.h>
#include <stdio.h>
#include "backtrace.h"

#ifdef NDEBUG
#define panic(message)
#else
/// @brief Causes the program to abort and print a message
///
/// @param [in] message The error message you wish to spit out to stderr.
#define panic(message)                                               \
do {                                                             \
fprintf(stderr, "panicked at: %s:%d\n", __FILE__, __LINE__); \
fprintf(stderr, "--> %s %s\n", message);                     \
fprintf(stderr, "--> STACKTRACE START\n");                   \
print_backtrace(2);                                          \
fprintf(stderr, "--> STACKTRACE END\n");                     \
fflush(stderr);                                              \
abort();                                                     \
} while (0)
#endif


Example usage would be like this

int some_function(uint8_t* data, size_t length) {
if (length == 0) {
panic("Zero length buffer provided!");
}

//
// consume data
//

return 1;
}


Output from this macro will look like the following.

panicked at: /home/warmwaffles/code/example/rbp_test.c:173
--> message: Failed to insert the block correctly
--> START STACKTRACE
bin/rbp_test(print_stacktrace+0x22)[0x403002]
bin/rbp_test(test_block_insertion+0x218)[0x402b68]
bin/rbp_test(main+0x2f)[0x402eef]
/usr/lib/libc.so.6(__libc_start_main+0xf1)[0x7f3e3cb5c291]
bin/rbp_test(_start+0x2a)[0x4027fa]
--> END STACKTRACE
Aborted (core dumped)


I wanted the core dump to take place so that I can inspect it if I need to. Luck favors the prepared, and I always like to be prepared.

## Assert

I use assert(expr) liberally through out my code to ensure that my program operates as I intend it to. Sometimes I make a mistake and would like to be notified where it happened and how deep in the call stack it did.

Unfortunately vanilla assert(expr) does not do this. But it is a simple enough macro to override and provide a little more meta information about where it failed and why.

/// @file core/assert.h
#pragma once

#include <stdlib.h>
#include <stdio.h>
#include "backtrace.h"

#ifdef NDEBUG
#define assert(expr)
#else
#define assert(expr)                                                                     \
if(!(expr)) {                                                                        \
fprintf(stderr, "assertion (%s) failed at: %s:%d\n", #expr, __FILE__, __LINE__); \
fprintf(stderr, "--> STACKTRACE START\n");                                       \
print_backtrace(2);                                                              \
fprintf(stderr, "--> STACKTRACE END\n");                                         \
fflush(stderr);                                                                  \
abort();                                                                         \
}
#endif



As you can see it looks almost exactly the same as panic(message) does. However, I want the expression to be spit out into stderr so that I can see what expression failed.

assertion (1 == 0) failed at: /home/warmwaffles/code/example.c:170
--> STACKTRACE START
bin/example(print_backtrace+0x22)[0x4015a2]
bin/example(test_block_insertion+0x1cc)[0x40134c]
bin/example(main+0x2f)[0x40148f]
/usr/lib/libc.so.6(__libc_start_main+0xf1)[0x7f989aaec291]
bin/example(_start+0x2a)[0x40102a]
--> STACKTRACE END
Aborted (core dumped)


Found these little bits of code to be useful, and figured others would probably find it useful as well.

Mixins are a bit of a touchy spot for me. I am in a love hate relationship when it comes to them. In some cases, they work brilliantly and in other cases they hide complexity.

I have a few rules that I try to follow when I am considering using mixins.

1. Does it hide complexity / indirection?
2. Will it benefit the code base to share code?
3. Will it override methods or will methods need to be overriden?

Moving methods into mixins to just move them is not sufficient enough to warrant the need for mixins. It is something that should be used to refactor once a pattern is established. Mixins should only be used for adding abilities to classes.

Let’s take a look at an example. We need an ability to revoke tokens mixed into three separate classes.

module Revokable
# Sets when the token was revoked
# @return [void]
def revoke
@revoked_at = Time.now
end

# Check to see if the token was revoked
# @return [TrueClass,FalseClass]
def revoked?
!!@revoked_at
end
end


The personal token represents a token that belongs to a user and never expires, but it can be revoked.

class PersonalToken
include Revokable

attr_accessor :id, :token, :user_id
end


The access token represents a token that is only available for a limited time.

class AccessToken
include Revokable

attr_accessor :id, :token, :user_id, :refresh_token_id

def expire
@expired_at = Time.now
end

def expired?
!!@expired
end
end


The refresh token never expires but is only used for getting another access token.

class RefreshToken
include Revokable

attr_accessor :id, :token, :user_id
end


We have accomplished is adding an ability to three classes without hiding a lot of complexity. If you find that your classes have too many methods defined, that should be a sign that it is too complex. But, do not immediately reach for mixins just because it makes the class less cluttered. It actually hides the mess as opposed to solving the issue.

Use mixins wisely!

Let me introduce you to my new library called Yukata. It is a light weight Ruby attribute library that is configurable and extendable.

Virtus has been a huge inspiration for this library. I enjoyed the DSL it offered, while allowing me to have a quick way to make data objects.

Here is an example on how to utilize Yukata:

class Person < Yukata::Base
attribute :first_name, String
attribute :last_name,  String
attribute :born_on,    DateTime
attribute :married,    Boolean, default: -> { false }
end


The #attribute method is straight forward with its meaning. It is dynamically creating both getter and setter methods for the object. It can be thought of as a fancy attr_accessor but with a few extra features. It provides a fast way to discover what data type can be expected for that attribute.

## Example Usage

When using Yukata, the the initializer expects a hash to be provided or a class that behaves like a Hash.

john = Person.new({
:first_name => 'John',
'last_name' => 'Doe',
:born_on => '1969-01-16T00:00:00+00:00'
})


Yukata will take the hash and assign the values to their respective attribute keys. If a setter method is defined, then a corresponding value can be passed as well.

class Foo < Yukata::Base
attr_accessor :bar
attribute :qux, String
attribute :baz, String, writer: false

def baz=(value)
@baz = value.to_s
end
end

foo = Foo.new({
bar: 'woot',
qux: 'herp',
baz: 'derp'
})

foo.bar # => 'woot'
foo.qux # => 'herp'
foo.bas # => 'derp'
foo.attributes # => { bar: 'woot', qux: 'herp' }


If a :coerce => false is passed, then Yukata will not attempt to coerce that attribute and leave it as is. This can be handy if a custom coercion is desired for the specific model. Here is an example:

class Episode < Yukata::Base
attribute :season, Integer
attribute :number, Integer
attribute :name,   String, coerce: false

# @override overides the yukata definition
def name=(value)
@name = '%sx%s - %s' % [@season, @number, value]
end
end

episode = Episode.new({ season: 1, number: 1 })
episode.name = 'Foo Bar'
episode.name # => '1x1 - Foo Bar'


Now, remember just because there is access directly to instance variables does not mean it is okay to abuse them. With great power comes great responsibility, this means I am not responsible for your mistakes.

## Setting Attribute Defaults

Sometimes the objects need default values if it is not set. Defaults are lazily loaded. They will only be set once the getter method is called.

class Book < Yukata::Base
attribute :name,       String
attribute :created_at, DateTime, default: -> { DateTime.now }
end


## Registering Custom Coercions

This library only comes with basic coercers. I tried to make as little assumptions about the data coming in as I could. I believe that the consumer of the library should be the one who defines the coercions.

If the value can not be coerced, it is simply passed through and left alone.

Yukata.coercer.register(String, Array) do |string, target|
string.split(' ')
end


When declaring an attribute, both the reader and writer can be skipped. There is a use case where this would be handy.

class Book < Yukata::Base
attribute :title, String, writer: false, reader: false

def title=(value)
@title = value.to_s
end

def title
@title
end
end


This is a bit contrived, but it demonstrates the following:

1. The expected return data type for #title is a String.
2. Custom coercer is defined.
3. The attribute will be included when #attributes is called on Book.

If :writer => false is provided, there would be no need to include :coerce => false since the coercion only takes place when the value is being set on the object.

## Conclusion

I wrote this library becaues I wanted to see how Virtus accomplished this task and how I could go about doing it differently. This is a highly configurable library that can be used to put your fat models on a diet.