Saturday, April 1, 2017

Golang: A Fast HTTP Mux with http.HandlerFunc

Objective

1. First we wanted a fast HTTP mux with http.HandlerFunc support
2. It should have good benchmarks

The search

Exploring at github.com/julienschmidt/go-http-routing-benchmark for a list of fast routers, we found one very interesting fast router => the Gin router.


Exploring Gin (github.com/gin-gonic/gin)

Gin have:
- very good benchmarks
- good middleware design
- and above all, it has impressive sets of unit tests

What to change in Gin

The only thing we don't like from Gin is its handlers.

     // This handler will match /user/john but will not match neither /user/ or /user  
      router.GET("/user/:name", func(c *gin.Context) {  
           name := c.Param("name")  
           c.String(http.StatusOK, "Hello %s", name)  
      })  

This is completely understandable because the "context" package was not part of Golang's "http" package prior to go1.7.


Replacing Gin's handlers

Request.WithContext(...) is slow.

Request.WithContext is doing a shallow copy of "request", which in my benchmarks costs about 300+ ns/op.
(ref: https://golang.org/pkg/net/http/#Request.WithContext)

Context alternative

So we try to find other means under the Request type definition:

 type Request struct {  
     Method string  
     URL *url.URL  
     Proto   string   
     ProtoMajor int  
     ProtoMinor int  
     Header Header  
     Body io.ReadCloser  
     GetBody func() (io.ReadCloser, error)  
     ContentLength int64  
     TransferEncoding []string  
     Close bool  
     Host string  
     Form url.Values  
     PostForm url.Values  
     MultipartForm *multipart.Form  
     Trailer Header  
     RemoteAddr string  
     RequestURI string  
     TLS *tls.ConnectionState  
     Cancel <-chan struct{}  
     Response *Response  
 }  


Instinctively, Form and PostForm are quick choices. But you will end up running ParseForm() to initialize them.

Other interesting options are Body and GetBody.

But Body is used widely within the http package,  and we don't want to break it.

The final choice is "GetBody". We can hide our Gin Context inside of GetBody without breakage (hopefully). And benchmarks gives us about 150+ ns/op or  20+ ns/op.

Here are the changes done:
https://github.com/noypi/router/commit/82586b8967933e131ecfa7d1a2eeaadb2ade0e78


Thanks,
S

Monday, March 20, 2017

Golang: Quick and Easier Leveled Logging

using logfngithub.com/noypi/logfn

Example usage:
 DBG("subject=%v, tag=%v, row=%v", subject, tag, row)  

 // print stacktrace (last 8 calls)  
 DBG.PrintStackTrace(8)  

 // similar to PrintLn()  
 DBG.Ln(var1, var2, var3)  


first lets define our LogFunc:
 type LogFunc func(fmt string, params ...interface{})  


then our hero type the LogLevel:
 type LogLevel int  


our goal is to have something like:
 var (  
      CRITICAL = g_logLevel.WrapFunc(int(LogCritical), log.Printf, "[pkgname][C] ")  
      ERR      = g_logLevel.WrapFunc(int(LogError),    log.Printf, "[pkgname][E] ")  
      WARN     = g_logLevel.WrapFunc(int(LogWarning),  log.Printf, "[pkgname][W] ")  
      INFO     = g_logLevel.WrapFunc(int(LogInfo),     log.Printf, "[pkgname][I] ")  
      API      = g_logLevel.WrapFunc(int(LogApi),      log.Printf, "[pkgname][A] ")  
      DBG      = g_logLevel.WrapFunc(int(LogDebug),    log.Printf, "[pkgname][D] ")  
 )  


so, we wrap our LogFunc, with the code:
 func (this *LogLevel) WrapFunc(level int, fn LogFunc, prefix string) LogFunc {  
      prefix = strings.Replace(prefix, "%", "", -1)  
      return func(s string, as ...interface{}) {  
           if level <= int(*this) {  
                fn(prefix+s, as...)  
           }  
      }  
 }  


finally, a way to set the log level:
 func (this *LogLevel) SetLevel(n int) {  
      *this = (LogLevel)(n)  
 }  


we defined our levels as:
 const (  
      LogCritical LogLevel = iota  
      LogError  
      LogWarning  
      LogInfo  
      LogApi  
      LogDebug  
 )  

Thanks.

Sunday, March 19, 2017

Golang: Traverse a Map in Sorted Keys

Using mapk from github.com/noypi/mapk.

mapk gives you sorted keys everytime you use Each(...) or EachFrom(prefix, ...).

mapk is using slice:
type _kv struct {
    k, v interface{}
}
type _kvslist []*_kv

type _SliceMap struct {
    kvs _kvslist
    cmp func(k1, k2 interface{}) int
}

taking advantage of the "sort" package's powerful tools, like the Search() function:
func (this _SliceMap) find(k interface{}) int {
    return sort.Search(len(this.kvs), func(i int) bool {
        return 0 <= this.cmp(this.kvs[i].k, k)
    })
}

since Search() needs the container to be sorted, we sort the slice every time we add an element:
func (this *_SliceMap) Put(k, v interface{}) {
 if 0 == len(this.kvs) {
  this.kvs = append(this.kvs, &_kv{k: k, v: v})
  return
 }

 i := this.find(k)
 if i >= len(this.kvs) {
  this.kvs = append(this.kvs, &_kv{k: k, v: v})
  return
 }

 if 0 == this.cmp(this.kvs[i].k, k) {
  this.kvs[i].v = v
  return
 }

 this.kvs = append(this.kvs[:i], append(_kvslist{&_kv{k: k, v: v}}, this.kvs[i:]...)...)

}

here's a benchmark, comparing the native use of "map" and "gtreap":

BenchmarkPutTen_GTreap-8                     1000000       1932 ns/op      640 B/op       30 allocs/op
BenchmarkGetTen_GTreap-8                     1000000       1681 ns/op      480 B/op       20 allocs/op
BenchmarkEachFrom7of10_GTreap-8              5000000        291 ns/op       48 B/op        2 allocs/op
BenchmarkEachTen_GTreap-8                    5000000        354 ns/op       64 B/op        2 allocs/op
BenchmarkDeleteAdd5of10_GTreap-8              200000       5250 ns/op     3215 B/op       86 allocs/op
BenchmarkPutTen_Slice-8                      1000000       1768 ns/op      320 B/op       20 allocs/op
BenchmarkGetTen_Slice-8                      1000000       1167 ns/op      160 B/op       10 allocs/op
BenchmarkEachFrom7of10_Slice-8              10000000        134 ns/op       16 B/op        1 allocs/op
BenchmarkEachTen_Slice-8                    50000000         35.0 ns/op        0 B/op        0 allocs/op
BenchmarkDeleteAdd5of10_Slice-8              1000000       2012 ns/op      624 B/op       25 allocs/op
BenchmarkPutTen_Native-8                     5000000        361 ns/op        0 B/op        0 allocs/op
BenchmarkGetTen_Native-8                    10000000        187 ns/op        0 B/op        0 allocs/op
BenchmarkDeleteAdd5of10_Native-8             3000000        383 ns/op        0 B/op        0 allocs/op
BenchmarkEachFrom_Native_NOT_SUPPORTED-8    2000000000          0.00 ns/op        0 B/op        0 allocs/op
BenchmarkEachTen_Native-8                   10000000        153 ns/op        0 B/op        0 allocs/op

Thanks.

Monday, October 24, 2016

Go: Windows DLL

https://github.com/golang/go/issues/11058 workaround by @chai2010
go build -buildmode=c-archive -o libxxx.a gcc -m64 -shared -o xxx.dll xxx.def libxxx.a -Wl,--allow-multiple-definition -static -lstdc++ -lwinmm -lntdll -lWs2_32
Then use VS's lib command to generate xxx.lib:
lib /def:xxx.def /machine:x64

Monday, April 27, 2015