I thought I've written exactly the same blog post last year, but I guess I didn't. Maybe I was smart enough not to show this negative feeling.

It's almost the end of year. Every year, I've been thinking the same thing in this season and anxious. That is my skill, in one word. It might also be called my career path, knowledge or experience. I just don't know how I can describe what I'm worrying for.

Since 2004, I've been working as a software developer. There are couple of years of blank periods but most of my career is being Java developer, good or bad. When I started my career, I was just an ordinary newbie who had a bit of programming skill and assigned to the *deadline is already decided* project. Everything was new including somewhat deadline was there but no specification situation, and there were lots of things to learn, like how to capture screen shot and put it into excel sheet... At that moment, struts 1.0 was there and the concept of DI has just come. After 2 years, I quit the first job and became server administrator... sort of.

In year 2009, I've moved to The Netherlands. At that time, I had some blank as a Java developer. So I didn't know when CI is appeared. DI was still there but a bit brushed up. I could use Java annotation instead of writing huge application-context.xml. I still had lots of things to learn, however I just noticed that I was just learning the usage of libraries. If those libraries disappeared in 5 years, then this knowledge is just useless. Luckily, I've met Scheme and started writing own implementation to build some tools I can use on daily routine.

Writing a Scheme implementation gave me lots of challenges. I wasn't a good C programmer, well am still not, though. How function pointer works, how x86 works, how to retrieve stack area information, difference between POSIX and Windows, those were all new for me. I've never read C89 or C99 specification, so I will never be a C expert, but at least now I know where to look at if I need to find something.

If you write an implementation, then you also need to write libraries. Fortunately, or unfortunately, there's not much Scheme code. So I needed to write loads of code. If I see the code written in very first period of my Scheme life, those look ugly. I can't say I understand how Scheme works perfectly and I can write universal beautiful code. All what I know is how to write workable code and at least that works for me. Maybe I can say I also now the basic concept of paradigms which are used in Scheme programming, such as first class object and CPS.

7 years later, I'm still anxious. Maybe I feel like I know nothing essential. I know numbers of libraries and how those works. I know how to implement TLS or other network protocols. But are these essential? I only have vague idea what's essential knowledge is. It should be knowledge that I can use; even if I need to make software with a language I've never used; in the different IT field; after 20 years so on. And maybe I've already known that there's no such thing. And maybe the fact that I knew this reality makes me more anxious.

If I become a high skilled programmer, would this be gone? It might be better not to think about this and just study hard since I know I need to know more.




SagittariusはCommon Lispから移植したOAuth1.0を扱うライブラリを持っている。っが、このライブラリ将来OAuth2に対応させれるようにしたのかキーワード引数が大量にあり今一使い勝手が悪いい。っで、最近職場でOAuth2上に構築されたプロトコルの一つであるOpenID Connectを使うということが持ち上がっていることもあり、OAuth2のライブラリがいるかなぁという機運が高まってきている。

The OAuth 1.0 protocol ([RFC5849]), published as an informational document, was the result of a small ad hoc community effort. This Standards Track specification builds on the OAuth 1.0 deployment experience, as well as additional use cases and extensibility requirements gathered from the wider IETF community. The OAuth 2.0 protocol is not backward compatible with OAuth 1.0. The two versions may co-exist on the network, and implementations may choose to support both. However, it is the intention of this specification that new implementations support OAuth 2.0 as specified in this document and that OAuth 1.0 is used only to support existing deployments. The OAuth 2.0 protocol shares very few implementation details with the OAuth 1.0 protocol. Implementers familiar with OAuth 1.0 should approach this document without any assumptions as to its structure and details.
太字にした一文があるため、廃止されているとはいえ両方サポートした方がよさげな感じがしている。例えばWikipediaのList of OAuth providersのページによれば、OAuth1しかサポートしていないプロバイダーも結構ある(Bitbucket、Flickr、Tumbler辺りは個人的に使うかもしれない)。

  • (rfc oauth)
    • (rfc oauth v1.0)
      • client and so?
    • (rfc oauth v2.0)
      • client and so?
  • (rfc oauth1)
  • (rfc oauth2)
無難に分けた方がいい気がするなぁ、変に統合すると混乱しそうだし。現在ある(net oauth)ライブラリはそのままにするが、deprecatedという形にすればいいかな。うまいことやれるなら新しいライブラリを下請けにする形にしたいが、そこまで手が回るかは疑問。



この記事はLisp Advent Calendar 2016の19日目の記事です。

最近c.l.s.に面白い投稿があった。これである。 要約をすると、R7RSに以下の一文を追加しようという話になる。
It is an error to invoke the continuation of a top-level expression or the expression of a top-level definition more than once.
(define-library (foo)
  (import (scheme base))
  (export reload count)
    (define counter (vector 0))              ;; (1)
    (define (count) (vector-ref counter 0))  ;; (2)
    (define reload (call/cc (lambda (k) k))) ;; (3)
    (vector-set! counter 0 (+ 1 (count)))))  ;; (4)

(import (scheme base) (scheme write) (foo))  ;; (5)

(unless (>= (count) 10)
  (display (count))
  (reload reload))
;; end of the file


継続はSchemer以外のLisperには馴染みのない概念かもしれないので簡単に説明しておく。Schemeの仕様書を読むとなんとも面倒な言葉で書かれているが、厳密には多少違うが言ってしまえばコールグラフのことである(Clojure/Conjではスタックだと断言していた。まぁその通りである。)。 上記のコードでは以下のようにプログラムが実行される:
  (??) --> (1) --> (2) --> (3) --> (4) --> (??)



例えばこのプログラムが1ファイルに書かれていたとしよう。そして、処理系は一つの式を読み込み、実行という順序でファイルを処理したとする。この場合に(??)にくるのは次の式を読み込むという処理になるはずだ。そうすると、保存したreloadを起動した際に起きると予測されるのは、コメント;; end of the fileが読み込まれEOFが返されることであると言える(注:unlessから始まる式は既に読まれている)。ファイルの読み込みがEOFを返した場合、通常そのファイルは全て実行されたと解釈されるので、処理は終了するのが筋である。とすれば、この継続をこの位置で起動すると処理が終了すると言える。(必ずしも処理が終了するわけではないことに注意)

さて、c.l.s.の投稿ではLarcenyで上記のプログラムを走らせた場合想定通りに動いたとされている。これはどういうことか?簡単に言えばLarcenyはvan Tonderの展開器を使っているからとなる。もう少し突っ込んで解説をすると、van Tonderの展開器はR5RS上でポータブルにライブラリ機能を追加している。R5RSにはライブラリ機能は存在しないので実行時になんとかやっているわけだ。例えば上記のプログラムは概ね以下のように解釈される。
  (let ()
    (define counter ...)
    (define (count) ...)
    (define reload ...)
    (vector-set! ...)
    (register-library '(foo) ...)
  (resolve-import ...)
  (unless (>= (count) 10) ...))
このようにプログラムが展開された場合、(??)に当たる部分は何になるだろうか?そう、(register-library ...)の部分である。そして、プログラム全体が一つの式として解釈されているので、次の式を読み込む必要がない。これによりLarcenyでは見かけ上期待した結果が帰ってくるということになる。この問題はこの2つだけが予測される結末ではないのが面白いところである。





REPL上ではどうだろうか?REPLでは(vector-set! ...)が実行された後に次の式を読み込むために一旦制御が入力待ち状態になる。つまり、reloadの起動は(vector-set! ..)を実行し入力待ちの状態になるはずである(少なくともSagittarius上ではそうなる)。これももちろんREPLの実装依存になるだろう。





この記事はLisp Advent Calendarの2日目の記事です。



Working group 2 will develop specifications, documents, and proofs of practical implementability for a language that embodies the essential character of Scheme, that is large enough to address the practical needs of mainstream software development, and that can be extended and integrated with other systems.
The purpose of this work is to facilitate sharing of Scheme code. One goal is to be able to reuse code written in one conforming implementation in another conforming implementation with as little change as possible. Another goal is for users of this work to be able to understand each other's code based on a shared and unambiguous interpretation of its meaning.
The language is not necessarily intended for educational, research, or embedded use, though such uses are not prohibited. Therefore, it may be a "heavyweight" language compared to the language designed by working group 1.
From Charter for working group 2





  • SRFI 1 (scheme list)
  • SRFI 133 (scheme vector)
  • SRFI 132 (scheme sorting)
  • SRFI 113 (scheme set)
  • SRFI 114 (scheme set char)
  • SRFI 125 (scheme hash-table)
  • SRFI 116 (scheme list immutable)
  • SRFI 101 (scheme list random-access)
  • SRFI 134 (scheme deque immutable)
  • SRFI 135 (scheme textual)
  • SRFI 121 (scheme generator)
  • SRFI 128 (scheme lazy-seq)
  • SRFI 41 (scheme stream)
  • SRFI 111 (scheme box)
    ボックスSRFI。ボックスは名前が示す通り単なる箱で、Schemeのオブジェクトをなんでも格納しておくことができるもの。これを使うと明示的に参照渡しが可能になるはず。まぁ、あると便利だけど、なければ(vector obj)で代用できてしまうようなもの。コンパイラが頑張ると最適化してくれるかもしれない。
  • SRFI 117 (scheme list queue)
  • SRFI 124 (scheme ephemeron)
    実装者泣かせSRFIの一つ。キーと値を持つデータ構造で、基本的な部分では弱い参照と一緒なんだけど、値がキーを参照していた際でもGCされるという特徴を持つもの。噂によるとまともに実装されているのはRacketとMIT Schemeくらいらしい。
  • SRFI 128 (scheme comparator)


  •  OrangeDocket: 数値関係
    • Numeric types and operations: 
      • Integer division: SRFI-141
      • Bitwise integer operations: SRFI-142, SRFI-60 or R6RS
      • Fixnums: SRFI-143 or R6RS
      • Flonums: SRFI-144 or R6RS
      • Compnums: John's proposal
      • Random numbers: SRFI-27 plus Gauche's generator
      • Prime numbers: Gauche's prime number library
    • Numeric and semi-numeric data structures
    • Enumerations (なんでこれここにあるんだろう?)
    • Formatting (同上)
  • YellowDocket: 構文関係
    • Low-level macros (有名どころの低レベルマクロがリストされてる)
    • Syntax parameters: SRFI-139
    • Matching (パターンマッチ)
    • Combinators
    • Conditional procedures (if等をdefineで定義した感じのなにか?)
    • cond guards: SRFI-61
    • Lambda* (Scheme Workshop 2013に出た論文がもとっぽい)
    • Void (eq?で比較可能なNULLオブジェクト)
    • Named parameters: John's proposal or SRFI-89
    • Generalized set!: SRFI-17
    • and-let*: SRFI2
    • receive: SRFI-8
    • cut/cute: SRFI 26
    • Loops: SRFI-42, foof-loop or Chibh loop
    • Generic accessors/mutators: SRFI-123
    • Conditions (例外、なぜにR6RSではないのか…)
  • GreenDocket: ポータブルに書けない何か
    • File I/O
    • Threads: SRFI-18
    • Real-time threads: SRFI-21 (分ける必要あるのか?)
    • Socket: SRFI-106
    • Datagram channels (UDP sockets)
    • Timers: SRFI-120 (なんでこれがあるんだろう?)
    • Mutable environments
    • Simple POSIX (Windowsは完全無視ですね、わかります)
    • Access to the REPL (これ完全にR6RS殺しに来てるな…)
    • Library declarations
    • Symbols
    • Interfaces
    • Process ports (紛らわしいけど、プロセスI/Oをポートにリダイレクトする話)
    • Directory ports (ディレクトリをポートみたいに読む、正直紛らわしい)
    • Directory creation/deletion
    • Port operations
    • System commands (Process portsと被り気味な気がする)
    • Pure delay/force (スレッドを使ったdelay/forceっぽい)
  • BlueDocket: ポータブルだけど高度なものたち(portable but advanced things)
    • Time types: SRFI-19
    • Binary I/O
    • Character conversion (要するにR6RSのコーデック的なもの)
    • Parallel promises (pure delay/forceじゃだめなん?)
    • Pathname objects
    • URI objects
    • Unicode character database
    • Environment (紛らわしいけどusernameとかメモリとかを引くためのもの)
    • Trees (リストにあるけど提案なし)


Syntax parameter 2

Couple of months ago, I've written about syntax parameter. Now, I noticed that this implementation isn't really useful in certain (probably most of the) cases. For example, consider like this situation: You want to parameterise user inputs and your macro need to cooperate with auxiliary macros something like this:
;; default input is '()
(define-syntax-parameter *input* (syntax-rules () ((_) '())))
(define-syntax aux
  (syntax-rules ()
    ((_ body ...)
     (let ((user-input (*input*)))
       ;; do what you need to do
       body ...))))
(define-syntax foo
  (syntax-rules (*input*)
    ((_ (*input* alist) body ...)
     (syntax-parameterize ((*input* (syntax-rules () ((_) 'alist))))
       (foo body ...)))
    ((_ body ...) (aux body ...))))

(foo (*input* ((a b) (c d))) 'ok)
;; *input* is '()
I would expect that in this case *input* should return ((a b) (c d)). And, indeed, Racket returns ((a b) (c d)) as I expected.

The problem is simple. My implementation of syntax-parameterize is expanded to letrec-syntax with bound keyword. However since the above case crosses macro boundaries, *input* always refers to globally bound identifier. Now, my question is; can it be implemented portable way? If I read "Keeping it Clean with Syntax Parameters", then it says it's implemented in a Racket specific way. (If I read it properly, then syntax parameter is really a parameter object which can work on macro.)

Hmmm, I need to think how I can implement it.











Connection and session model

CAUTION: It's rubbish. Just my brain storming.

I'm not even sure if this is a proper model name or not. But I often see in low level communication specification, there is a connection and sessions (or channels) belong to the connection. Sometimes (or most of the time?) channels can be recovered on different connections, but here I discuss non recoverable channel for my ease.

I've written couple of scripts which handles this type of things already. For example AMQP. I think one of the pros for this model is preventing actual connection count. Recently, I've faced a concurrency problem caused by reading data from one single connection. At that moment, I didn't want to create much of connections (but in the end realised that there's no other way, though). Then I've started thinking about it.

Now I'm thinking can this be done in generic way or at least something which can be reused instead of writing the same mode everywhere? Suppose,a connection have a port. If a session is derived from the connection, then connection gives the session a session id or so. Whenever data are read from sessions, then actual connection dispatches read data according to the required session id. Something like this:
(import (rnrs))

(define-record-type connection
  (fields port ;; suppose this is input/output port
  (protocol (lambda (p)
              (lambda (port)
                (p port (make-eqv-hashtable))))))
(define-record-type session
  (fields id connection))

(define (open-connection port) (make-connection port))
(define (open-session connection)
  (let* ((sessions (connection-sessions connection))
         ;; FIXME
         (id (+ (hashtable-size sessions) 1))
         (session (make-session id connection)))
    (hashtable-set! sessions id session)

(define (read-from-session session)
   (session-connection session)
   (session-id session)))
To keep it clear, here connection is the physical connection and sessions are logical layers derived from a connection. A channel is a communication pipe between connection and session.

Obviously, this doesn't work as it is even if I implement reading part, because:
  1. There's no way to know which message should go which session
  2. There's no negotiation how to open a session, this type of things are usually client-server model.
To know how to dispatch messages, connections need to know how message frames look like. So connections should have something like receive-frame field which contains a procedure receiving a message frame?
(define-record-type connection
  (fields port
          ;; suppose this returns 2 values, message id and payload
  (protocol (lambda (p)
              (lambda (port receive-frame negotiate-session)
                (p port receive-frame  negotiate-session (make-eqv-hashtable))))))
Is it good enough? But what if there are like these 2 process is simultaneously running; opening a session and reading payload from a session. So it might be a good idea to have channel like this?
(define-record-type connection
  (fields port
          ;; takes session-id, channel-id and connection.
          ;; returns physical-session-id
  (protocol (lambda (p)
              (lambda (port receive-frame negotiate-session)
                (p port receive-frame
(define-record-type session
  (fields id channel-id connection
          (mutable physical-session-id)))
(define-record-type channel
  (fields id buffer))

(define (open-connection port) (make-connection port))
(define (open-session connection)
  (let* ((sessions (connection-sessions connection))
         (channels (connection-channels connection))
         ;; FIXME
         (session-id (+ (hashtable-size sessions) 1))
         (channel-id (+ (hashtable-size channels) 1))
         (channel (make-channel channel-id
                                ;; IMPLEMENT ME
         (session (make-session session-id channel-id connection #f)))
    (hashtable-set! sessions session-id session)
    (hashtable-set! channels channel-id channel)
Hmmm, would it be enough? I'm not sure yet. And I'm not even sure if this is useful. Maybe I need to implement a prototype.




ライブラリは(text sxml object-builder)としてみた。基本的な使い方はこんな感じ。
(import (rnrs)
        (text sxml ssax)
        (text sxml object-builder)
        (srfi :26))

(define-record-type family
  (parent xml-object))
(define-record-type parents
  (parent xml-object))
(define-record-type child
  (parent xml-object))
(define-record-type grand-child
  (parent xml-object))

(define family-builder
   (family make-family
    (parents make-parents
     (? child make-child
        (? grand-child make-grand-child))))))

(define sxml (call-with-input-file "family.xml" (cut ssax:xml->sxml <> '())))

(sxml->object sxml family-builder)
;; -> family object
<?xml version="1.0" ?>
  <parents father="Daddy" mother="Mommy">
    <child name="Big bro">
      <grand-child name="Bekkie">Boo!</grand-child>
      <grand-child name="Kees">Bah!</grand-child>
    <child name="Sis"/>
sxml-object-builderマクロはなんとなくいい感じにオブジェクトビルダーを構築してくれる。基本的には(tagname constructor . next-builder)みたいな構文。next-builderが空リストだと渡ってきたSXMLそのまま使う感じ。後は量指定子だったり、複数タグにしたり。最終的にはXSDと同等の表現力を持てたらなぁとは思っている(ほぼカバーしてるつもりだけど、仕様書を真面目に読んだことがないので何が足りてないのかが分かっていないという…)。








  • Chicken
    • Eggが便利
    • Cに変換すれば高速に動く
    • コミュニティが大きい
  • Chez
    • 現状最高速の処理系
  • Guile
    • GNU公式拡張言語
    • コミュニティが大きい
  • IronScheme
    • .Netで動く
  • Larceny
    • 高速
    • R7RSハイブリッド
  • Racket
    • 高速
    • コミュニティが大きい
    • Planetが便利
    • リッチな環境がついてくる(DrRacket)
  • Mosh
  • Ypsilon 
  • Vicare
    • ファンが多い?
  • Chibi
    • コンパクト
    • C拡張が書きやすい
  • Gauche
    • 日本語ドキュメント
    • 手軽?
  • Kawa
    • JVMで動く







  • R6RS/R7RSハイブリッド
  • リーダーマクロ
  • そこそこ手軽
  • Cトランスレータ有り(バイナリ走らせるのにランタイムがいるけど)



C translator (2)

Starting 0.7.8, Sagittarius provides experimental C translator. Currently it's more or less toy quality (though, just using is not a problem). There are 2 way of translations:
  1. Dump all required library cache
  2. Doing the same as precompiled files
Both solutions have problems. The first one requires huge amount of static area and generated files are huge. The second one can't translate macros.

Dumping cache file is not a good solution so I want to discard it. To do so, second solution must support macro translation. The challenge of doing it is that macro contains shared structures and may contain subrs (Scheme procedures defined in C). Translating shared structure to C may not be so difficult but subrs. Subrs themselves don't have any information where it's defined, so they only have their name and C pointer. As far as I know, there's no subr in macro so this may not be a problem as long as users (me mostly) don't do some black magic (and I know I can...).

One of the benefits (or maybe this is the only) of translating C is that considerably short amount of starting time. Sagittarius is, unfortunately, not so fast implementation, and its starting time is not so fast even when it loads only cached libraries. For example, one of my daily routine scripts takes 500ms to just showing help message (8000ms if it's not cached) and translated version takes 130ms. It's not that much difference (yet) but still it's almost 5 times faster than raw Scheme.

However, I think it's still slow or doing some unnecessary things. Consider the following script:
(import (rnrs))

(define (main args)
  (display args) (newline))
If I run this script, then it loads 32 libraries currently. But the only bindings required in this script are display and newline. So if the C translator is smart enough to detect them, then the result script should look like this:
;; imaginary script... 
(define (main args)
  (#{(core)#display} args) (#{(core)#newline}))
Then it doesn't require any unnecessary library loading.

Hmm, I think I have a lot to do.


Scheme Workshop 2016 奈良






【Nash: a tracing JIT for Extension Language】

【Ghosts in the machine】

【R7RS update】
AlexによるR7RSの近況。まだ見ぬSRFI-142とSRFI-143が並んでいたのでArthurで止まっているのか、Johnがまだ提出してないのかは謎。Red Docketは終わったらしいのだが、正式な発表はまだされてない気がするけど、c.l.sで見逃したかな?


【Function compose, Type cut, And the Algebra of logic】

【Multi-purpose web framework design based on websocket over HTTP Gateway】


【miniAdapton: A Minimal Implementation of Incremental Computation in Scheme】
AdaptonというMemoriseのテクニックに一つをSchemeで実装した話。入力ツリーの一部が変更されても共通部分は再利用しているような感じだった。リポジトリも公開されてるし、ソース見た方が早いかな。しかし、代理の発表(論文の共著者であるが) なのにえらくうまいことやるなぁと前回見た時も思ったなぁ。あれくらいうまくしゃべれるようになりたい(なんか違う)。

【Deriving Pure, Functional One-Pass Operations for Processing Tail-Aligned Lists】





(import (srfi :18) (util logging) (util file))

(define (print-log logger)
  (trace-log logger "trace")
  (debug-log logger "debug")
  (info-log logger "info")
  (warn-log logger "warn")
  (error-log logger "error")
  (fatal-log logger "fatal")
  (terminate-logger! logger))

(print-log (make-logger +info-level+ (make-appender "[~l] ~w4 ~m")))
(print-log (make-async-logger +debug-level+ 
         (make-appender "[~l] ~w4 ~m")
         (make-file-appender "[~l] ~w4 ~m" "log.log")))

(print (file->string "log.log"))
[info] 2016-08-30T16:02:29+0200 info
[warn] 2016-08-30T16:02:29+0200 warn
[error] 2016-08-30T16:02:29+0200 error
[fatal] 2016-08-30T16:02:29+0200 fatal

[debug] 2016-08-30T16:02:29+0200 debug
[info] 2016-08-30T16:02:29+0200 info
[warn] 2016-08-30T16:02:29+0200 warn
[error] 2016-08-30T16:02:29+0200 error
[fatal] 2016-08-30T16:02:29+0200 fatal
[debug] 2016-08-30T16:02:29+0200 debug
[info] 2016-08-30T16:02:29+0200 info
[warn] 2016-08-30T16:02:29+0200 warn
[error] 2016-08-30T16:02:29+0200 error
[fatal] 2016-08-30T16:02:29+0200 fatal

make-async-loggerはスレッドを一つ消費する代わりにログの書き出しをバックグラウンドでやってくれる。アペンダーが複数あったり、処理が重い(メール送るとか) 等あるときに役に立つと思う。



Ephemeron and reference barrier

Recently, there's the post on SRFI-124 ML about reference barrier. The SRFI doesn't require reference-barrier procedure due to the non-trivial work to implement. Now, there's a post which shows how to implement portably, like this:
(define last-reference 0)
(define (reference-barrier x)
  (let ((y last-reference))
    (set! last-reference x)
It seems ok, but is it? The SRFI says like this:
This procedure ensures that the garbage collector does not break an ephemeron containing an unreferenced key before a certain point in a program. The program can invoke a reference barrier on the key by calling this procedure, which guarantees that even if the program does not use the key, it will be considered strongly reachable until after reference-barrier returns.
Due to the lack of knowledge and imagination, I can't imagine how it should work precisely on like this situation:
(let loop ()
  (when (some-condition)
    (reference-barrier key1)
    (reference-barrier key2)
Should key1 and key2 be guaranteed not to be garbage collected or only the last one (key2 in this case)? If the first case should be applied, then the proposed implementation doesn't work since it can only hold one reference. To me, it's rather rational to hold multiple reference since it's not always only one key needs to be preserved. However if you extend to hold multiple values, then how do you release the references without calling explicit release procedure? If the references would not be released, then ephemerons would also not release its keys. Thus calling reference-barrier causes eternal preservation.

Suppose this assumption is correct, then reference-barrier should work like alloca with automated push so that the pushed reference is on the location where garbage collector can see them as live objects. In this sense, allocated references are released automatically once caller of reference-barrier is returned and multiple references can be pushed. Though, it's indeed non-trivial task to implement. I hope there's no errata which says reference-barrier is *not* optional, otherwise it would be either a huge challenge or dropping the support...


暗黙の総称関数 (部分解決編)


(import (rnrs) (clos user))

(let ()
  (define-generic foo)
foo ;; -> &undefined

(import (rnrs) (clos user))

(let ((foo 'foo))
  (define-method foo (o) o)
  (print (foo 1)))
;; -> error

(let ((a 1) (a 2)) a) ;; -> error
(let (("a" 1)) 1)     ;; -> error
(let ()
  (define foo)
  (define-generic foo)
  (define-method foo (o) o)
  (foo 1))
;; ???



Implicit generic function creation

define-method would create a generic function implicitly if there's none defined yet. It's convenient and I've written libraries (probably only one) depending on this behaviour. (c.f. (binary data))

Convenience would usually be a trade-off  of consistency, at least in my case. For example, this is a long standing bug (though, I've just issued):
(import (clos user))

(let ()
  (define-generic foo))
;; -> should be &undefined
This is because define-method would create a generic function during macro expansion, and it would be an unexpected result in this case if it didn't:
  (define-generic foo)
  (define-method foo (o) #t))
In this case, define-method should not make implicit generic function, but macro expansions are done in the same compilation time as define-generic. Thus, it's impossible to know if it should create or not. To let define-method know it shouldn't, define-generic also inserts binding into current environment (a library) during macro expansion.

Can't I do better? Creating global binding during macro expansion is rather ugly, but I don't I can get rid of it (or maybe the future?). Yet, I think I can avoid to create unwanted one like above example. It's still just an idea but if define-generic and define-method can see if they are used in a scope, then it seems there's a way. Since Sagittarius has current-usage-env and current-macro-env procedures, it is possible to access compile time environment during macro expansion. Thus, it should even be able to detect whether or not define-method should create an implicit generic function or not.

This should work, let's see.



$ ./scheme2c -o out.c foo.scm
$ gcc `sagittarius-config -L -I -l` -O2 out.c
中身はほぼなんでもいいんだけど、引数を受け取るにはSRFI-22のmainがないとたぶんうまいこと使えないはず(command-line手続きで引数が取れない)。 ちなみに出来上がりCファイルは超巨大になる可能性がある。参考までに、このスクリプト自信をCに変換したら45MBのファイルになった(64bit環境)。ファイルが巨大になるのはこのスクリプトが何をやっているかを見ればすぐにわかるのだが、端的に言うとキャッシュファイルを引っ張ってきてCのバイト列にしてるから。

なんでこんなものを書いたかというと、特に理由はないんだけど、バイナリ一個で動くといいかなぁと思ったから。今のところランタイムとして最低でも DLL か .so (OSXなら .dylib) がいるがこの辺はそのうち気が向いたら何とかする予定。

キャッシュをそのままダンプしているので、性能的なメリットは一切ない。強いて言えば多少ファイルアクセスが減るくらい。今のところsagittarius-configは Windows 版にはつけてないので VC でバイナリを作りたい場合は多少の工夫が必要。ちなみに、インストール時にパスが決まるからインストーラでやらないといけないというのがついてない理由。










流石にこれはパフォーマンスに影響がでて、例えば(rfc http)のように大量に他のライブラリ(数えたら131個あった)に依存するものをキャッシュなしで読み込むと大体10~15%程度パフォーマンスが落ちた。Cでは手続き内でやっていた処理がScheme側に移動したことで手続き呼び出しに変わったこと等によるオーバーヘッドだろうなぁとは思っているものの、まぁしょうがないか。キャッシュになってさえいればほぼ変わらないので(キャッシュはCなので当たり前だが)、初回起動のみが遅いと思えばそこまで気にすることでもないかなぁ。そもそもコンパイルとマクロ展開が遅いでそっちを先に何とかしないとという話である。(text sql)のコンパイルとか環境によっては10秒かかるし…2000行以上にわたるマクロを10秒で展開すると思えばそこまで悪くないともいえるのか?っがコンパイル待ってる間にコーヒー飲み終わる勢いなのは流石になぁ…






書いてる途中でSaitoAtsushiさんの実装の存在を思い出してライブラリの名前を確認したら、(rfc websocket)とまるかぶりだった。Pegasus用のformulaまで書いていただいているのにこのままぶつけるのもなぁと思い確認してみたところ
とのことだったので、遠慮なくぶつけさせていただいた。他の名前の候補としては、(rfc :6455 websocket)とか(rfc websockets)(Cのlibwebsocketsに倣って)とかあったけど、「普通の名前」(RFCの名前)を使うことにした。

(import (rnrs) (rfc websocket))

;; Creates WebSocket object
(define websocket (make-websocket "wss://echo.websocket.org"))

;; Sets event handlers
(websocket-on-open websocket
  (lambda (ws) (display 'CONNECTED) (newline)))
(websocket-on-text-message websocket
  (lambda (ws text) (display text) (newline)))

;; Connects to the server
(websocket-open websocket)

;; Sends a message
(websocket-send websocket "Hello")

;; Close it
(websocket-close websocket)
    (websocket-on-open (make-websocket "wss://echo.websocket.org")
     (lambda (ws) (display 'CONNECTED) (newline)))
    (lambda (ws text) (display text) (newline))))
どっちがいいかは好みだろうけど(流石に下のはあまり使わないか?)。ドラフトのまま絶賛放置中(期限切れてるから破棄されてるの?)のWebSocket over HTTP/2にも頑張れば対応できるようにはしてある(プラグイン書くだけ)。






  • Cでwith-exception-handlerを使っているか
    • 使ってなかった。なんでこれC側にあるんだろう状態だった。
  • C側で例外投げたらどうなるの
    • この記事の肝、気になったら続きを読んで。


Call stack
  +----------+  +---------+  +---------+  +---------+
--| Sg_Apply |--| C func1 |--| C func2 |--| C func3 |
  +----------+  +---------+  +---------+  +---------+
                                             C error
   PC = CALL
   | Return to previous frame |
   PC = RET
   | Return to calling raise  | --> PC to return = CALL raise
   | Return To previous frame |



Exception handling in C world

The following piece of code goes into infinite loop on Sagittarius.
(import (rnrs))
 (lambda (k) #t)
 (lambda ()
   (guard (e (#f #f))
     (error 'test "msg"))))
I couldn't figure it out why this never returned at first glance. And it turned out to be very interesting problem.

The problem is related to continuation boundary and exception handlers. If you print the k, then only the very first time you'd see error object with "msg" after that it'd be "attempt to return from C continuation boundary.". But why?

This happens when you capture a continuation and invoke it in the different C level apply. Each time C level apply is called, then VM put a boundary mark on its stack to synchronise the C level stack and VM stack. This is needed because there's no way to restore C level stack when the function is returned. When a continuation is captured on the C apply which is already returned, then invocation of this continuation would cause unexpected result. To avoid this, the VM checks if the continuation is captured on the same C level apply.

Still, why this combination causes this? The answer is raise is implemented in C world and exception handlers are invoked by C level apply. The whole step of the infinite loop is like this:
  1. guard without else clause captures a continuation.
  2. When error is called, then guard re-raise and invoke the captured continuation. (required by R6RS/R7RS)
  3. Exception handler is invoked by C level apply.
  4. VM check the continuation boundary and raises an error on the same dynamic environment as #3
  5. Goes to #3. Hurray!
There are 2 things I can do:
  1. Create a specific condition and when with-exception-handler received it, then it wouldn't restore the exception handler. [AdHoc]
  2. Let raise use the Scheme level apply. (Big task)
#1 probably wouldn't work properly. #2 is really a huge task. I need to think about it.





  • リモートREPL用のリーダーを作る
    • 面倒
  • リモートREPL用ポートを作って、リードマクロを読ませる
    • アドホックだけど、悪くない気がする


  • ライブラリファイルの変更を検知したらリロードする機構
  • 依存関係の親が子を知る手段


Syntax parameters (aka SRFI-139)

Marc Nieper-Wißkirchen submitted an interesting SRFI. The SRFI was based on the paper 'Keeping it Clean with Syntax Parameters'. The paper mentioned some corner case of breaking hygiene with datum->syntax, which I faced before (if I knew this paper at that moment!).

In 'Implementation' section, the SRFI mentions that this is implemented on 'Rapid Scheme', Guile and Racket. Unfortunately there's no portable implementation. In my very prejudiced perspective, Guile isn't so fascinated by macro, so the syntax parameters is probably implemented on top of existing macro expander (I haven't check since Guile is released under GPL, and if I see the code it may violates the license). If so, it might be able to be implemented on syntax-case.

Without any deep thinking, I've written the very sloppy implementation like this:
(library (srfi :139 syntax-parameters)
  (export define-syntax-parameter
  (import (rnrs))

(define-syntax define-syntax-parameter
  (syntax-rules ()
    ((_ keyword transformer)
     (define-syntax keyword transformer))))

(define-syntax syntax-parameterize
  (lambda (x)
    (define (rewrite k body keys)
      (syntax-case body ()
        (() '())
        ((a . d)
         #`(#,(rewrite k #'a keys). #,(rewrite k #'d keys)))
        (#(e ...)
         #`#(#,@(rewrite k #'(e ...) keys)))
         (and (identifier? #'e)
              (exists (lambda (o) (free-identifier=? #'e o)) keys))
         (datum->syntax k (syntax->datum #'e)))
        (e #'e)))
    (syntax-case x ()
      ((k ((keyword spec) ...) body1 body* ...)
       (with-syntax (((n* ...)
                      (map (lambda (n) (datum->syntax #'k (syntax->datum n)))
                           #'(keyword ...)))
                     ((nb1 nb* ...)
                      (rewrite #'k #'(body1 body* ...) #'(keyword ...))))
         #'(letrec-syntax ((n* spec) ...) nb1 nb* ...))))))
And this can be used like this (taken from example of the SRFI):
(import (rnrs) (srfi :139 syntax-parameters))

(define-syntax-parameter abort
  (syntax-rules ()
    ((_ . _)
     (syntax-error "abort used outside of a loop"))))

(define-syntax forever
  (syntax-rules ()
    ((forever body1 body2 ...)
      (lambda (escape) 
    (syntax-rules ()
      ((abort value (... ...))
       (escape value (... ...))))))
  (let loop ()
    body1 body2 ... (loop))))))))

(define i 0)
 (display i)
 (set! i (+ 1 i))
 (when (= i 10)

(define-syntax-parameter return
  (syntax-rules ()
    ((_ . _)
     (syntax-error "return used outside of a lambda^"))))

(define-syntax lambda^
  (syntax-rules ()
    ((lambda^ formals body1 body2 ...)
     (lambda formals
 (lambda (escape)
      (syntax-rules ()
        ((return value (... ...))
  (escape value (... ...))))))
    body1 body2 ...)))))))

(define product
  (lambda^ (list)
    (fold-left (lambda (n o)
   (if (zero? n)
       (return 0)
       (* n o)))
        1 list)))

(display (product '(1 2 3 4 5))) (newline)
I've tested on Chez, Larceny, Mosh and Sagittarius.

This implementation violates some of 'MUST' specified in the SRFI.
  1. keyword bound on syntax-parameterize doesn't have to be syntax parameter. (on the SRFI it MUST be)
  2. keyword on syntax-parameterize doesn't have to have binding.
And these are the sloppy part:
  1. define-syntax-parameter does nothing
  2. syntax-parameterize traverses the given expression.
If there's concrete test cases and the above implementation passes all, I might send it as a sample implementation for R6RS.


Weirdness of self evaluating vector

Scheme's vector has a history of being non-self evaluating datum and self evaluating datum. The first one is on R6RS, and the latter one is R7RS (not sure about R5RS). Most of the time, you don't really care about the difference other than it requires ' (quote) or not. However, you may need to think about the difference and maybe also think self evaluating causes more trouble. One of the particular case (and this is the only case I think self evaluating vector is evil) is when vector is used in macro.

Have a look at this case:
(import (rnrs))

(define-syntax foo
  (syntax-case ()
    ((_ e) e)))

(foo #(a b c))
What do you think how it behaves? The answer is depending on the standard. On R6RS, vectors are not self evaluating data so this should be an error. So you can't complain if you'd get a daemon from your nose. On R7RS (of course you should change the importing library name to (scheme base)), on the other hand, vectors are self evaluating data so this should return the input vector.

Now, how about this case?
(import (scheme base) (scheme write))

(define-syntax foo
  (syntax-rules ()
    ((_ "go" (v ...) ())         #(v ...))
    ((_ "go" (v ...) (e e* ...)) (foo "go" (v ... t) (e* ...)))
    ((_ e ...)                   (foo "go" () (e ...)))))

(foo a b c d e)
What would be the expansion result of macro foo? I think this is totally up to implementations (if it's not please let me know). For example, Chibi returns vector of something like {Sc #22 #<Environment 4365836288> () t} (syntactic closure, I think), Sagittarius returns vector of identifier, and Larceny returns vector of symbol t. If you put ' (quote) to the result template then the expansion result should be the same as Larceny returns (though, Chibi still returned a vector of syntactic closure, so this might not be defined, either).

Back to the first case. The first case sometimes bites me when I write/use R6RS macro in R7RS context. For example, SRFI-64 is implemented in R6RS macro and using it like this:
(import (scheme base) (srfi 64))

(test-begin "foo")

(test-equal "boom!" #(a b) (vector 'a 'b))
;; FAIL!!

On Sagittarius, R6RS macro transformer first converts all symbols into identifiers, then syntax information will be stripped only if expressions have quote. Now, SRFI-64 is implemented on the R6RS macro transformer and the vector doesn't have quote. Thus, symbols inside of the vector are converted to identifiers. If it's R6RS, then it's an error. But if it's R7RS, it should be a valid script.

I have sort of solution (not sure if I do it or not): Internally, symbol and the identifiers converted from symbols without any context (c.f. not using datum->syntax) are theoretically the same. So if compiler sees such an identifier, then it should be able to unwrap it safely.

I haven't decided how it should be. So for now, just a memo and let it sleep.




  • その会社で絶対働きたいという熱意がいる
    • 会社名のタトゥーをいれるくらいとか
    • 40時間越えて働いても残業代請求しないとか
  • 金が欲しい開発者ならGoogleにでも行け
    • 熱意があるやつが欲しいそうな
  • 給料が高すぎるとよくないから(全社員共通の)上限がある
    • ちなみに上限はAmazonのオファーより1万5千ユーロほど低かった
    • そして上限を上げるつもりはないらしい(といわれた)
  • オファーを比較して自分の市場価値を探るのは好きではない
    • 自分もやってたけど意味無いってさ
    • 比較せずさっさと受けろという意味だとは思うけど
  • 副業をしなければいけないほど給料が安い
  • そうでないのなら、Google並みの給料もらってもやってると思う


  • 労働を提供する対価を雇用者に求めている
  • 仕事のやりがいと称して対価を下げる行為を嫌っている
  • 対価に勝る評価方法はない
  • 僕の労働をより評価してくれる雇用者に容易に移る






WG2の議論でレコードの健全性について出てる(c.f. record types)。個人的にレコードが暗黙的にアクセサを作るかどうかというのはとりあえずどうでもよくて(R6RSでは明示されなければ暗黙的に作るってなってるし)、健全性のテストコードが問題だった。







Because it's fun

Couple of days ago, I've had a job interview, and one of the interviewer said something very interesting. I don't remember exact sentence but something like this: If I make a framework for hobby, it's okay. I didn't understand what the purpose of this comment, so I said "if there's no framework then no choice, right?" Then, he said "if you need to use this framework for work, then you need to consider a lot of things such as buffer overflow.".  Well, sort of agree and sort of disagree.

The reason why I needed to make loads of framework is basically because nobody would make other than me. If it's major language such as Java, then you just need to google it and find something. But I'm using Scheme, more specifically Sagittarius. Sagittarius, unfortunately, doesn't have many libraries. Of course, I'm trying to write something useful, but there's only one resource so it doesn't increase the number drastically. And even more unfortunate thing is that it's not so popular so there's not many users. If number of users is small, then not many libraries are written. (It's rather my fault because I still think it only needs to fit my hand and didn't advertise much...) Then if you need something, you gotta write it.

The part I agree with the opinion is the cost of coding. If there are well maintained libraries for you purpose, using them would reduce some time to redevelop the same functionality. Especially if the library is mature enough, then it takes a lot of resource to make your own implementation such level. (e.g. Spring Framework or so)

But should it only be like this? If you are a programmer, you want to write something from scratch because it's fun, don't you? I hear almost every year new trend framework. Not sure the actual motivations are, might be unmatched framework, might be just for fun, etc. And most of the time, it has some bugs. What I want to say is not something like new frameworks are buggy, but they can be famous even the initial version is buggy. So it's pity if you don't write/show something useful because it might not be perfect.

The very first version may only confirm your own requirement. I usually write such libraries and just put it on GitHub (you'll find loads of junks on my GitHub repository). After using it, I usually notice what's missing or not considered. If you have choice to write whatever you want to, then don't hesitate to write especially under the reason of  "buffer overflow". (It's more for me, though)





先に結論を書くと、Javaは極めて冗長に書く必要があるのでプロトタイプ的なものを作るには向かない。(ので、体力勝負系のコーディングテストに持ってこられると非常に面倒。) 成果はGithubかBitbucketに置けと言われたので、晒しても問題ないと判断して晒す(まさかプライベートリポジトリに置けって意味ではないと思うし)。とりあえず以下はなんとなくな比較

そうは言っても割と大きめな時間の差があると個人的には思う。言語の習熟度とか、ライブラリ習熟度の差なのかもしれないけど、一応これでも職業Java屋歴10年以上あるのでそこまでの差はないとしたいところ(むしろScheme歴は6-7年なのでJava歴より短い)。個人的に最も生産性(ここでは時間のことを指す)に響いたのはREPLの存在だったと思う。SchemeではリモートREPLを使ってサーバに変更を即時反映させて挙動を確認していたのに対し、Javaでは毎回ビルドするという切ない状況だったのは大きい。対話的に確認できることの偉大さというのを改めて実感した気がする。次いで大きかったのはデザインパターンの有無。JavaだとなんとなくGeneric DAOパターンを使わないといけないかなぁという脅迫観念に襲われたので、とりあえずそれを使ったんだけど、このパターンは恐ろしく生産性が低い。あらかじめあるものを使うのならばいいのだが、エンティティ毎にDAO書いてサービス書いてとかやってるもんだから、非常に面倒だった。(この辺IDEがワンクリックでやってくれると違うのかもしれない。)細かくボディーブローのように効いたのはJavaの1クラス1ファイルの仕様や、キーボードから指が離れる瞬間が多いこと。別にEmacs最高とかいうつもりはないけど、こういうところで地味に時間を取られた気がする。



File monitoring on OS X using FSEvents

(sagittarius filewatch) on OS X is using kqueue (2) currently. Using kqueue isn't bad just having couple of limitation such as no capability of directory monitoring (due to my laziness). It is OK on BSD environment since this is the only choice to do it. However, on OS X, there are FSEvents APIs which allow users to monitor filesystem. When I research it, it doesn't require loads of file descriptors nor limit file/directory only monitoring. So I thought this might be a good underlying implementation for OS X and implemented like this.

If it ends without any problem, as usual, I don't write any blog post. Yes, there's a huge problem. It doesn't allow me to write tail emulator. I first thought my implementation has an issue. So I've written this small piece of code to check if it works as my expect.
#include <CoreServices/CoreServices.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>

static void callback(ConstFSEventStreamRef stream,
                     void *callbackInfo,
                     size_t numEvents,
                     void *evPaths,
                     const FSEventStreamEventFlags evFlags[],
                     const FSEventStreamEventId evIds[])
  FILE *fp = (FILE *)callbackInfo;
  char buf[1024];
  const char **paths = (const char **)evPaths;
  for (int i = 0; i < numEvents; i++) {
    while (1) {
      int n = fread(buf, 1, sizeof(buf), fp);
      fwrite(buf, 1, n, stdout);
      if (feof(fp)) break;

int main(int argc, char **args)
  if (argc != 2) {
    fputs("fstail file", stderr);
  FILE *fp = fopen(args[1], "r");
  fseek(fp, 0, SEEK_END);
  CFStringRef s = CFStringCreateWithCString(kCFAllocatorDefault, args[1],
  CFArrayRef ar = CFArrayCreate(NULL, (const void **)&s, 1, NULL);
  FSEventStreamContext ctx = {0, fp, NULL, NULL, NULL};
  int flags = kFSEventStreamCreateFlagFileEvents;
  FSEventStreamRef stream = FSEventStreamCreate(NULL, &callback, &ctx, ar,
                                                0, flags);
  FSEventStreamScheduleWithRunLoop(stream, CFRunLoopGetCurrent(),
  return 0;
This didn't work like tail command, unfortunately. It didn't receive any event after it got first event. If you get a technical problem, you probably search Google or Stack Overflow. Yes, I've found the very similar issue: No callback get called from FSEventStreamCreate with modifications created by self in watched file. It seems the question didn't get any useful answer. Feeling like I'm missing something, but I don't know what. I've also changed latency argument to non zero value but got no luck.

As long as this problem is not solved, I can't use FSEvents. So for now, I use kqueue, which works perfectly fine for my purpose, on OS X...




  1. eqv?が循環構造のあるレコードを受け取るとSEGVを起こす。
  2. equal?がレコードの中身を比較する(R6RS的には規格違反)



じゃあどうしたか?実装をR6RSのequal?とそれ以外という風に分けた。ポータブルなコードを書くという上ではR6RSの方がより細かく規定してあるのでいいのだろうけど、これくらいの処理系拡張は許して欲しいという気持ちもある。そうすると実装を二つにする以外に方法が思いつかなかったのだ。ということで、(rnrs base)(scheme base)で定義されているequal?は別物になった。この動作に依存したコードを書くというのはあんまりないと思うけど、ハッシュテーブルをequal?で作ってキーにレコードを使用している場合は影響がでるという話。


肉体改造部 第十七週



  • 体重: 70.4kg (-0.5kg)
  • 体脂肪率: 21.6% (-0.8%)
  • 筋肉率:43.3% (+0.1%)





  • レベル0:その言語を全く話せない
  • レベル5:日常会話レベル、かなり大変だがその言語で生活できる
  • レベル10:ビジネスレベル、職場でコミュニケーションができる
  • レベル15:高等教育レベル、日本なら高校卒業時の国語
  • レベル20:専門家レベル、この言語に関する知識で飯が食える

  • 日本語:レベル15(多分もう少し低いが、敬語とか忘れたし、日本の高校卒業してるということで)
  • 英語:レベル13 - 14(多少色眼鏡付きな気もするけど)
  • オランダ語:レベル4 - 6(一応生きていけるが辛い。仕事では使えない)

全体の習熟度とすればこんな感じなんだろうけど、個別にみると意外と面白いことが分かる。例えばくしゃみをした人にかける言葉として英語では「Bless you」、オランダ語では「Gezondheid」がある。オランダに長く住んでいるので誰かがくしゃみをすると、たとえくしゃみをした人がオランダ語を話せなくても、「Gezondheid」というようになった。同様なものに「Alstublieft」もしくは「Alsjeblieft」がある。もう少し込み入った例だと、「Kan ik pinnen?」ある。これは「Can I use debit?」のオランダ語バージョンと思ってもらえばいいのだが、アメリカに行ったときとか、店員さんがオランダ語喋れない場合でもこれが勝手に出てくる(アメリカで出た際は流石に、「Kan ik,,, can I use credit card?」になったが)。特定のシチュエーションに於いてはアウトプットが最も多いものが勝手に口をついてくるみたいである。

  1. グループ内に日本人以外がいる
  2. カタカナ語を英語の発音で喋ってしまう



Inter-operable hidden binding

The subject might sound weird, but I couldn't find any better name. So please bare with it.


Suppose you have a situation that 2 macros need to refer the same implicitly bound variable. For example, consider define-generic and define-method; the declaration of generic function is done by define-generic, and adding definition to it is done by define-method. Now, you want to write it as simple as possible, so you've decided to use implicitly bound hashtable.
;; naive definition of define-generic (doesn't work)
(define-syntax define-generic-naive
  (syntax-rules ()
    ((_ name)
       (define implicit (make-eq-hashtable))
       (define (name . args)
          ;; lookup and execute

(define-syntax define-method-naive
  (syntax-rules ()
    ((_ name formals body ...)
       (define (real-proc . formals) body ...)
       (define dummy
         ;; oops, implicit can't be referred here!
         (hashtable-set! implicit 'name real-proc))))))
Now, your task is make this happen somewhat.

Passing explicitly

Taking this path isn't really what I want, but it's the only way to do it on R7RS.
(define-syntax define-generic-explicit
  (syntax-rules ()
    ((_ name table)
       (define table (make-eq-hashtable))
       (define (name . args)
          ;; lookup and execute

(define-syntax define-method-naive
  (syntax-rules ()
    ((_ name table formals body ...)
       (define (real-proc . formals) body ...)
       (define dummy
         (hashtable-set! table 'name real-proc))))))
The problem with this implementation is that you need to know the name of the shared bindings. It might be good for debugging or breaking, but you probably don't want to care about something only used internally.

Macro generating macro

If 2 macros cannot refer the variable defined in one of the macro, then make it in the one macro like this:
(define-syntax define-generic/defmethod
  (syntax-rules ()
    ((_ name method-name)
       (define shared (make-eq-hashtable))
       (define (name . args) 
         ;; lookup and execute
       (define-syntax method-name
         (syntax-rules ()
           ((_ name shared formals body (... ...))
              (define (real-proc . formals) body (... ...))
              (define dummy 
                (hashtable-set! shared 'name real-proc))))))))))
It's probably better than explicitly passing, but it's rather ugly. The method definition should be more generic. In this implementation, the method definition belongs to specific generic function definition.

Identifier macro

If you are using R6RS, then syntax-case can handle non list macro (not sure how it should be called, but I say identifier macro). So if the name of generic function itself can be evaluated to implicit definition name, then we can share the binding by referring the name.
(define-syntax define-generic
  (syntax-rules ()
    ((_ name shared)
       (define shared (make-eq-hashtable))
       (define (real-proc . args)
         ;; lookup
       (define-syntax name
         (lambda (x)
           (syntax-case x ()
             ((_ args (... ...)) #'(real-proc args (... ...)))
             (k (identifier? #'k) #'shared))))))))

(define-syntax define-method
  (syntax-rules ()
    ((_ name formals body ...)
       (define (real . formals) body ...)
       ;; method name should generic function name;
       ;; thus, it's an identifier macro to return
       ;; implicit table name.
       (define dummy (hashtable-set! name 'name real))))))
Better, at least for me. If I see it with half eye closed, then it looks like fake LISP-2.

Pitfalls I've got

I first thought that maybe I can use datum->syntax to create the same name; however, this wasn't a good idea. It is okay to if both generic function declaration and method definitions are in the same library; otherwise, you'd get a problem. Suppose you have a library (a) contains only define-generic and other library (b) contains define-method. Now, which template identifier you should use to generate the same name of the implicit binding? You need to use the identifier define-generic in the library (a), and it's impossible to use it in library (b). (This is the reason why I needed to write the version 2, macro generating macro.)


I don't have any intention to say, macro is the best, or something like that, but if something beyond procedure (in this case, emulating LISP-2, kind of), then it is rather necessary feature.


Generic record copy

I've found a tweet that says R7RS define-record-type doesn't create copier (or copy constructor) by default. Well, I can imagine why it doesn't if I think of C++'s copy constructor (which I think very confusing and causing unexpected behaviour). And it's also context dependent what exactly copy means.

Now, if I just say like this, then it's not so fun. So let's write kind of generic copy procedure. Before that, here our definition of copy is deep copy. So it creates a new object without having the same object inside. So more like cloning.


If it's generic, then it should work also builtin types. Generally, Scheme chose to have distinct procedures per types and what we want is one generic procedure. The very simple strategy would be dispatching. It might be convenient if users can specify how copy works per types. So the interface of copy procedure would look like this:
(define *copier-table* '())

(define (generic-copy obj)
  (cond ((assoc obj *copier-table* (lambda (x p) (p x))) =>
         (lambda (s) ((cdr s) obj)))
        ;; shallow copy, sort of
        (else obj)))

(define (register-copier! pred copier)
  (set! *copier-table* (cons (cons pred copier) *copier-table*)))
To register built-in types, we can do like this:
(register-copier! pair? list-copy)
(register-copier! vector? vector-copy)
(register-copier! string? string-copy)
(register-copier! bytevector? bytevector-copy)
Now, we have generic copy procedure for built-in types.

NB: list-copy and vector-copy doesn't consider the elements of copying object. If you want to follow the definition of copy here, you need to create own copy procedure.


You know how define-record-type works, right? It needs to be fed name of constructors, predicate procedures. So doing the same for copy procedure. Let's call our brand new record definition syntax define-record-type/copy. It would look like this:
(define-record-type/copy pare (kons a d) pare? pare-copy
   (a kar)
   (d kdr))
The extra argument pare-copy is the procedure automatically generated by the macro.

Implementation strategy

Now, how can we implement it? The strategy I chose (and probably this is the only way to do it portably) is that:
  • Collect field value and order it by constructor tag
  • Create object by passing above value with specified constructor
  • Set field values of fields which are not listed on constructor
So my implementation is like this:
(define-syntax define-record-copier
  (syntax-rules ()
    ((define-record-copier "emit" name (ctr f ...) (acc ...) ((a m) ...))
     ;; now we have all information
     (define (name obj)
       (let ((c (ctr (acc obj) ...)))
         ;; mutate if mutators are defined, then we use it.
         ;; to make it simple, we do for all mutator. so some
         ;; of them are just useless.
         ;; FIXME this is not efficient.
         (m c (a obj)) ...
    ((_ "mutator" name ctr accessor mutator ())
     (define-record-copier "emit" name ctr accessor mutator))
    ((_ "mutator" name ctr accessor (mutator* ...) ((f a) rest ...))
     (define-record-copier "mutator" name ctr accessor
       (mutator* ...) (rest ...)))
    ((_ "mutator" name ctr accessor (mutator* ...) ((f a m) rest ...))
     (define-record-copier "mutator" name ctr accessor
       (mutator* ... (a m)) (rest ...)))
    ((_ "collect" name ctr (acc ...) () (def* ...))
     (define-record-copier "mutator" name ctr (acc ...) ()(def* ...)))
    ((_ "collect" name ctr (acc ...) (field field* ...) (def* ...))
       ;; this part is not R7RS portable since 'foo' doesn't have to be
       ;; renamed (right?). so some of implementation may raise an error
       ;; of redefinition (e.g. Foment)
       ;; however we can't use letrec-syntax because it creates a scope.
       ;; sucks...
       (define-syntax foo
         (syntax-rules (field)
           ((_ ?n ?c
               ((field ac . ignore) rest (... ...))
               (next (... ...))
               (src  (... ...)))
            (define-record-copier "collect" ?n ?c (acc ... ac)
              (next (... ...)) (src (... ...))))
           ((_ ?n ?c (_ rest (... ...)) (next (... ...)) (src (... ...)))
            (foo ?n ?c (rest (... ...)) (next (... ...)) (src (... ...))))))
       (foo name ctr (def* ...) (field* ...) (def* ...))))
    ((_ name ctr (ctr-field* ...) (field-def* ...))
     (define-record-copier "collect" name ctr
       () ;; accessor
       (ctr-field* ...)
       (field-def* ...)))))

(define-syntax define-record-type/copy
  (syntax-rules ()
    ((_ name (ctr field* ...) pred copier field-def* ...)
       (define-record-type name (ctr field* ...) pred
         field-def* ...)
       (define-record-copier copier (ctr field* ...)
         (field* ...) (field-def* ...))))))
I usually use letrec-syntax to detect free identifier (well, it should be bound identifier but I don't think there's no way to do it in range of R7RS). But needed to use define-syntax (see comment).

Then you can use it like this:
(define-record-type/copy pare (kons a d) pare? pare-copy
   (a kar)
   (d kdr)
   (s pare-src pare-src-set!))

(register-copier! pare? pare-copy)
(let ((p (kons 'a 'b)))
  (pare-src-set! p '(src))
  (let ((c (generic-copy p)))
    (print (kar c))
    (print (kdr c))
    (print (pare-src c))))
(Write your own print procedure :P). The implementation is not efficient since we call mutator procedure no matter what. To make it efficient, you need to get mutators of which are not listed on constructor tags.

The whole scripts are here:


Use R6RS or SRFI-99.


肉体改造部 第十四週



  • 体重: 70.9kg (-0.3kg)
  • 体脂肪率: 22.4% (+0.3%)
  • 筋肉率:43.2% (±0.0%)








  1. pthread_getattr_npの呼び出し
  2. pthread_attr_getstackの呼び出し
  3. pthread_attr_destroyの呼び出し

    GC_API int GC_CALL GC_get_stack_base(struct GC_stack_base *sb)
      void * _tlsbase;

      __asm__ ("movl %%fs:4, %0"
               : "=r" (_tlsbase));
      sb -> mem_base = _tlsbase;
      return GC_SUCCESS;




サーバーが正常に立ち上がったかどうかを確認するのに起動ログを見るか、実際にアクセスして動かないことを確認するしかないというのがだるくなった。なので、ファイルを監視しつつ失敗のキーワードがあれば通知するものを作ったのだが、どうもプロセスをデタッチすると何も出力されないことに気付いた。Sagittarius 0.7.2までは子プロセスの標準入出力は常にパイプが割り当てられるのだが、親プロセスが終了するとパイプから出力を読み取るプロセスがなくなるので何も出力されないという話だった。これでは不便だなぁと思い、えいや!っと出力先を制御できるようにしてみた。

(import (rnrs) (sagittarius process))
(let ((proc (make-process "foo" '("process")))
      (outfile "pout"))
  (process-call proc :output outfile))



Ellipses expansion of syntax-rules

An interesting post was posted on c.l.s. (c.f. Nested ellipses) It's about how ellipses of syntax-rules should be expanded. The code is as follows:
(define-syntax test
  (syntax-rules ()
    ((test (x ...) ((y ...) ...) )
     '((x (x y) ...) ...) ) ) )
(test (a b c)
      ((1 2 3) (4 5 6) (7 8 9)) )
I'm not sure if this is an error since (x (x y) ...) contains 2 times x followed by an ellipsis. (I think it is, and SRFI-72 expander, a.k.a Van Tonder expander, signals an error.) So I've removed the first x and tested on couple of R6RS and R7RS implementations.
;; Removed the first x
(define-syntax test
  (syntax-rules ()
    ((test (x ...) ((y ...) ...) )
     '(((x y) ...) ...) ) ) )
(test (a b c)
      ((1 2 3) (4 5 6) (7 8 9)) )
(((a 1) (b 2) (c 3))
 ((a 4) (b 5) (c 6))
 ((a 7) (b 8) (c 9)))
(((a 1) (a 2) (a 3))
 ((b 4) (b 5) (b 6))
 ((c 7) (c 8) (c 9))) 
Implementations emit the #1 are the following:
  • All R6RS implementations
  • Foment
Implementations emit the #2 are the following:
  • Chibi
  • Sagittarius using (scheme base) library
  • Gauche
  • Picrin
To me, if I modify the template like above, it should emit the #1 result. Both R6RS and R7RS have some kind of specification of this pattern:
Pattern variables that occur in subpatterns followed by one or more ellipses may occur only in subtemplates that are followed by (at least) as many ellipses. These pattern variables are replaced in the output by the input subforms to which they are bound, distributed as specified.
R6RS: 11.19 - Macro transformers
Pattern variables that occur in subpatterns followed by one or more instances of the identifier ellipsis are allowed only in subtemplates that are followed by as many instances of ellipsis . They are replaced in the output by all of the elements they match in the input, distributed as indicated.
R7RS: 4.3.2 - Pattern language
I think the difference between R6RS and R7RS is matched ellipses consuming part. R6RS also requires the following:
The subtemplate must contain at least one pattern variable from a subpattern followed by an ellipsis, and for at least one such pattern variable, the subtemplate must be followed by exactly as many ellipses as the subpattern in which the pattern variable appears. (Otherwise, the expander would not be able to determine how many times the subform should be repeated in the output.)

This, I believe, restricts that input expressions of the multiple ellipses have the same length of input. In above example, x and y should have the same length. R7RS, on the other hand, requires to consume all input. Thus, x and y may have different length of inputs (e.g. (test (a b c) ((1 2 3 10) (4 5 6) (7 8 9))) should be valid on above example). In such cases, the expander can't determine how it should be expanded if it needs to expand like R6RS does (as R6RS mentioned).

Maybe there's more direct statement which specifies the behaviour of this case.


肉体改造部 第十二週



  • 体重: 71,2kg (+1.7kg)
  • 体脂肪率: 22.1% (+1.6%)
  • 筋肉率:43.2% (+0.6%)



ファイルシステムの監視 実装編

;; tail.scm
(import (rnrs) (getopt) (sagittarius filewatch) (binary io))

(define (tail file offset)
  (define watcher (make-filesystem-watcher))
  (define in (open-file-input-port file))
  ;; dump contents to stdout
  (define (dump)
    (let loop ()
      (let ((line (get-line in)))
        (unless (eof-object? line) 
          (put-bytevector (standard-output-port) line)
          (put-bytevector (standard-output-port) #vu8(10))
  (define size (file-size-in-bytes file))
  ;; move port position if the size if more than offset
  (when (> size offset) (set-port-position! in (- size offset)))
  ;; dump first
  ;; add path to file watcher
  (filesystem-watcher-add-path! watcher file '(modify) 
                                (lambda (path event) (dump)))
  ;; monitor on foreground.
  (filesystem-watcher-start-monitoring! watcher :background #f))

;; this tail is not line oriented
;; it shows tail of the file from the given offset.
(define (main args)
  (with-args (cdr args)
      ((offset (#\o "offset") #t "1024")
       . rest)
    (tail (car rest) (string->number offset))))
sash tail.scm foo


Windowsはファイルの監視ができないんだけど、ディレクトリの監視をすればどのファイルが変更されたかの情報が取れるので特に問題なかった。ありそうなのは、Windows Vista以降ではデフォルトでアクセス時間の変更がされないので、それの監視をしたい場合はシステムを弄らないといけないことか(実装とは関係ない) 。











(import (rnrs) (srfi :27))

(define zun "ズン")
(define doko "ドコ")
(define kiyoshi "キ・ヨ・シ!")

(define (zun? o) (string=? zun o))
(define (doko? o) (string=? doko o))
(define (kiyoshi! o) (display o) (display kiyoshi) (exit 0))

(define zun&doko (vector zun doko))
(define (zundoko-generator) (vector-ref zun&doko (random-integer 2)))

(define init-state 0)
(define (gen-next n) (lambda (o) (display o) n))
(define ->init (gen-next init-state))
(define states
  `#((,zun?  ,(gen-next 1) ,->init)
     (,zun?  ,(gen-next 2) ,->init)
     (,zun?  ,(gen-next 3) ,->init)
     (,zun?  ,(gen-next 4) ,->init)
     (,doko? ,kiyoshi! ,(gen-next 4)) ;; more than 4 zun, loop it

(random-source-randomize! default-random-source)

(let loop ((ns init-state))
  (let ((token (zundoko-generator))
        (state (vector-ref states ns)))
    (if ((car state) token)
        (loop ((cadr state) token))
        (loop ((caddr state) token)))))




  • 自分で探す
  • 向こうから声をかけられる
  • エージェント経由
  1. エージェントから連絡がくる
  2. 先方との面接日時を決める
    1. 面接
  3. エージェント経由でフィードバックを受け取る
  4. 採用、不採用が決まるまで2-3までを繰り返す








肉体改造部 第九週



  • 体重: 72.9kg (+0.1kg)
  • 体脂肪率: 23.7% (+0.1%)
  • 筋肉率:42.6% (±0.0%)




I'm currently working on ORM library (this) and have figured out that creating prepared statement is more expensive than I expected. You might be curious  how much more expensive? Here is the simple benchmark script I've used.
(import (rnrs)
        (sagittarius control)

(define conn (make-postgresql-connection
              "localhost" "5432" #f "postgres" "postgres"))
;; prepare the environment
(postgresql-open-connection! conn)
(postgresql-login! conn)
(guard (e (else #t)) (postgresql-execute-sql! conn "drop table test"))
(guard (e (else #t))
  (postgresql-execute-sql! conn "create table test (data bytea)"))

(postgresql-terminate! conn)

;; let's do some benchmark
(postgresql-open-connection! conn)
(postgresql-login! conn)

(define data
  (call-with-input-file "bench.scm" get-bytevector-all :transcoder #f))

(define (insert-it p)
  (postgresql-bind-parameters! p data)
  (postgresql-execute! p))

;; Re-using prepared statement
(let ((p (postgresql-prepared-statement
          conn "insert into test (data) values ($1)")))
  (time (dotimes (i 10) (insert-it p)))
  (postgresql-close-prepared-statement! p))

(define (create-it)
  (let ((p (postgresql-prepared-statement
            conn "insert into test (data) values ($1)")))
    (insert-it p)
    (postgresql-close-prepared-statement! p)))
;; Creating prepared statement each time
(time (dotimes (i 10) (create-it)))

;; bye bye
(postgresql-terminate! conn)
I'm using (postgresql) library.  <ad>BTW, I think this is the only portable library that can access database. So you gotta check it out. </ad> It's simply inserting the same binary data (in this case the script file itself) 10 times. One is re-using prepared statement, the other one is creating it each time. The difference is the following:
$ sash bench.scm

;;  (dotimes (i 10) (insert-it p))
;;  0.760319 real    0.008487 user    3.34e-40 sys

;;  (dotimes (i 10) (create-it))
;;  1.597769 real    0.014841 user    5.76e-40 sys
More than double. It's just doing 10 iterations but this much difference. (Please ignore the fact that the library itself is already slow.) There are probably couple of reasons including PostgreSQL itself but from the library perspective, sending messages to the server would be slow. Wherever a DB server is, even localhost, communication between script and the server is done via socket. And calling postgresql-prepared-statement does at least 7 times of I/O (and 5 times for postgresql-close-prepared-statement!). So if I re-use it, then in total 120 times (12 x 10, of course) of I/O can be saved.

Now, my ORM library hides low level operations such as creating prepared statement, connection management, etc. (that's what ORM should do, isn't it?). So keeping prepared statement in users' script isn't an option. Especially, there's no guarantee that users woudl get the same connection each time they do some operation. So it's better to manage it on the framework.

Sagittarius has (cache lru) library, undocumented though, so first I thought I could use this. After modifying couple of lines and found out, no this isn't enough. The library only provides very simple cache mechanism. It even doesn't provide a way to get all objects inside the cache. It's okay if the object doesn't need any resource management, however prepared statements must be closed when it's no longer used. Plus, LRU may not be good enough for all situations so it might be better if users can specify which cache algorithm should be used.

There are variety of cache algorithms. Implementing all of them would take a bit time. So it's better to make a framework or interface of cache. The framework/interface should have the following properties:
  • Auto eviction and evict event handler
  • Limitation of storage size (unlimited as well)
  • A way to get all cached objects
  • Implementation independent interface
Now, make myself busy.