FEED Validator

Congratulations!

This is a valid Atom 1.0 feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

• line 1, column 118: Use of unknown namespace: http://schemas.google.com/blogger/2008 [help]

  ... r.com/styles/atom.css" type="text/css"?><feed xmlns='http://www.w3.org/2 ...
                                               ^

• line 1, column 118: Unable to validate namespace: http://schemas.google.com/g/2005. See the Google Data specification at https://developers.google.com/gdata/docs/1.0/elements [help]

  ... r.com/styles/atom.css" type="text/css"?><feed xmlns='http://www.w3.org/2 ...
                                               ^

• line 1, column 1311: Self reference doesn't match document location [help]

  ... 31701633643336/posts/default?alt=atom'/><link rel='alternate' type='text ...
                                               ^

• line 448, column 0: content should not contain imageanchor attribute (19 occurrences) [help]

  <a href="https://www.bing.com/th?id=A4703f29460ecbae785b6b801e5b59d1 ...
  

• line 448, column 0: content should not contain data-original-height attribute (11 occurrences) [help]

  <a href="https://www.bing.com/th?id=A4703f29460ecbae785b6b801e5b59d1 ...
  

• line 448, column 0: content should not contain data-original-width attribute (11 occurrences) [help]

  <a href="https://www.bing.com/th?id=A4703f29460ecbae785b6b801e5b59d1 ...
  

• line 542, column 0: style attribute contains potentially dangerous content: break-before (5 occurrences) [help]

  <blockquote class="tr_bq" style="break-before: page; font- ...
  

• line 1095, column 0: content should not contain iframe tag (2 occurrences) [help]

  <iframe width="320" height="266" class="YOUTUBE- ...
  

Source: http://gdamore.blogspot.com/atom.xml

1. <?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?><feed xmlns='http://www.w3.org/2005/Atom' xmlns:openSearch='http://a9.com/-/spec/opensearchrss/1.0/' xmlns:blogger='http://schemas.google.com/blogger/2008' xmlns:georss='http://www.georss.org/georss' xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr='http://purl.org/syndication/thread/1.0'><id>tag:blogger.com,1999:blog-7528831701633643336</id><updated>2024-10-12T23:15:58.631-07:00</updated><category term="-"/><category term="--"/><category term="afe GLDv3"/><category term="atom"/><category term="blogger"/><category term="feedburner"/><category term="illumos arcmsr"/><category term="illumos gsoc"/><category term="joyent illumos zfs"/><category term="mxfe GLDv3"/><category term="oracle opensolaris"/><category term="rss"/><category term="sdcard"/><category term="zfs nexenta fma"/><title type='text'>/dev/dump</title><subtitle type='html'>Stack backtraces from the mind of Garrett.  Symbolic debugger not included.</subtitle><link rel='http://schemas.google.com/g/2005#feed' type='application/atom+xml' href='http://garrett.damore.org/feeds/posts/default'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default?alt=atom'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/'/><link rel='hub' href='http://pubsubhubbub.appspot.com/'/><link rel='next' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default?alt=atom&amp;start-index=26&amp;max-results=25'/><author><name>Anonymous</name><uri>http://www.blogger.com/profile/15319934747521320351</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><generator version='7.00' uri='http://www.blogger.com'>Blogger</generator><openSearch:totalResults>352</openSearch:totalResults><openSearch:startIndex>1</openSearch:startIndex><openSearch:itemsPerPage>25</openSearch:itemsPerPage><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-2712453512608765114</id><published>2018-12-10T11:47:00.002-08:00</published><updated>2018-12-10T11:47:59.721-08:00</updated><title type='text'>Golang sync.Cond vs. Channel... </title><content type='html'>The backstory here is that mostly I love the &lt;a href=&quot;http://golang.org/&quot;&gt;Go&lt;/a&gt; programming language.&lt;br /&gt;
2. <br />
3. But I've been very dismayed by certain statements from some of the core Go team members about topics that have significant ramification for my concurrent application design.  Specifically, <a href="https://github.com/golang/go/issues/9578">bold statements</a> to the effect that "<a href="https://tour.golang.org/concurrency/2">channels</a>" are the way to write concurrent programs, and deemphasizing condition variables.  (In one case, there is even a <a href="https://github.com/golang/go/issues/21165">proposal</a> to remove condition variables entirely from Go2!)<br />
4. <br />
5. <h3>
6. The Go Position</h3>
7. <br />
8. Essentially, the Go team believes very strongly in a design principle that can be stated thusly:<br />
9. <br />
10. "<i style="color: #222222; font-family: Arial, sans-serif;">Do not communicate by sharing memory; instead, share memory by communicating."</i><br />
11. <br />
12. This design principle underlies the design of channels, which behave very much like UNIX pipes, although there are some very surprising semantics associated with channels, which I have found limiting over the years.  More on that below.<br />
13. <br />
14. Certainly, if you can avoid having shared memory state, but instead pass your entire state between cooperating parties, this leads to a simpler, lock free (sort of -- channels have their own locks under the hood!) design.  When your work is easily expressed as a pattern of pipelines, this is a better design.<br />
15. <br />
16. <h3>
17. The Real World</h3>
18. <br />
19. The problem is that sometimes (frequently in the real world) your design cannot be expressed this way.   Imagine a game engine, dealing with events from the network,  multiple players, input sources, physics, modeling, etc.  One simple design is to use a single engine model, with a single go routine, and have events come in via many channels.  Then you have to create a giant select loop to consume events.  This is typical of large event driven systems.<br />
20. <br />
21. There are some problems with this model.<br />
22. <br />
23. <br />
24. <ol>
25. <li> Adding channels dynamically just isn't really possible, because you have a single hard coded select loop.  Which means you can't always cope with changes in the real world.   (For example, if you have a channel for inputs, what happens when someone plugs in a new controller?)</li>
26. <li>Any processing that has to be done on your common state needs to be in that giant event loop.  For example, updates to lighting effects because of an in game event like a laser beam needs to know lots of things about the model -- the starting point of the laser beam, the position of any possible objects in the path of the laser, and so forth.  And then this can update the state model with things like whether the beam hit an object, causing a player kill, etc.</li>
27. <li>Consequently, it is somewhere between difficult and impossible to really engage multiple CPU cores in this model.  (Modern multithreaded games may have an event loop, but they will also make heavy use of locks to access shared state, in order to permit physics calculations and such to be done in parallel with other tasks.)</li>
28. </ol>
29. <br />
30. <br />
31. So in the real world, we sometimes have to share memory still.<br />
32. <br />
33. <h3>
34. Limitations of Channels</h3>
35. <br />
36. There are some other specific problems with channels as well.<br />
37. <br />
38. <br />
39. <ul>
40. <li>Closed channels cannot be closed again (panic if you do), and writing to a closed channel panics. </li>
41. <li>This means that you cannot easily use go channels with multiple writers.  Instead, you have to orchestrate closing the channel with some other outside synchronization primitive, such as a mutex and flag, or a wait group.)  This semantic also means that close() is <i>not idempotent</i>.  That's a really unfortunate design choice.</li>
42. <li>It's not possible to broadcast to multiple readers simultaneously with a channel other than by closing it.  For example, if I am going to want to wake a bunch of readers simultaneously (such as to notify multiple client applications about a significant change in a global status), I have no easy way to do that.  I either need to have separate channels for each waiter, or I need to hack together something else (for example adding a mutex, and allocating a fresh replacement channel each time I need to do a broadcast.  The mutex has to be used so that waiters know to rescan for a changed channel, and to ensure that if there are multiple signalers, I don't wake them all.)</li>
43. <li>Channels are <i>slow</i>.  More correctly, select with multiple channels is slow.  This means that designs where I have multiple potential "wakers" (for different events) require the use of separate channels, with separate cases within a select statement.  For performance sensitive work, the cost of adding a single additional case to a select statement was found to be quite measurable.</li>
44. </ul>
45. There are other things about channels that are unfortunate (for example no way to peek, or to return an object to channel), but not necessarily fatal.<br />
46. <br />
47. What does concern me is the false belief that I think the Go maintainers are expressing, that channels are a kind of panacea for concurrency problems.<br />
48. <br />
49. Can you convert any program that uses shared state into one that uses channels instead?  Probably.<br />
50. <br />
51. Would you want to?  No.  For many kinds of problems, the constructs you have to create to make this work, such as passing around channels of channels, allocating new channels for each operation, etc. are fundamentally harder to understand, less performant, and more fragile than a simpler design making use of a single mutex and a condition variable would be.<br />
52. <br />
53. <a href="https://kaviraj.me/understanding-condition-variable-in-go/">Others</a> have written on this as well.<br />
54. <br />
55. <h3>
56. Channels Are Not A Universal Cure</h3>
57. <div>
58. <br /></div>
59. It <a href="http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/">has been said before</a> that the Go folks are guilty of ignoring the work that has been done in operating systems for the past several decades (or maybe rather of being guilty of NIH). I believe that the attempt to push channels as the solution over all others is another sign of this.  We (in the operating system development community) have ample experience using threads (true concurrency), mutexes, and condition variables to solve large numbers of problems with real concurrency for <i>decades, and doing so scalably</i>. <br />
60. <br />
61. It takes a lot of hubris for the Golang team to say we've all been doing it wrong the entire time.  Indeed, if you look for condition variables in the implementation of the standard Go APIs, <a href="https://github.com/golang/net/blob/e514e69ffb8bc3c76a71ae40de0118d794855992/http2/pipe.go#L18">you</a> <a href="https://github.com/golang/go/blob/42257a262c94d839364113f2dbf4057731971fc1/src/runtime/os_darwin.go#L30">will</a> <a href="https://github.com/golang/crypto/blob/a49355c7e3f8fe157a85be2f77e6e269a0f89602/ssh/buffer.go#L17">find</a> <a href="https://github.com/golang/leveldb/blob/259d9253d71996b7778a3efb4144fe4892342b18/leveldb.go#L76">them</a>.  Really, this is a tool in the toolbox, and a useful one, and I personally find it a bit insulting that the Go team seems to treat this as a tool with sharp edges with which I can't really be trusted.<br />
62. <br />
63. I also think there is a recurring disease in our industry to try to find a single approach as a silver bullet for all problems -- and this is definitely a case in point.  Mature software engineers understand that there are many different problems, and different tools to solve them, and should be trusted to understand when a certain tool is or is not appropriate.<br />
64. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/2712453512608765114/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=2712453512608765114' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2712453512608765114'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2712453512608765114'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/12/golang-synccond-vs-channel.html' title='Golang sync.Cond vs. Channel... '/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-3737755355297793857</id><published>2018-12-01T11:33:00.001-08:00</published><updated>2018-12-01T11:33:41.950-08:00</updated><title type='text'>Go modules, so much promise, so much busted</title><content type='html'>Folks who follow me may know that &lt;a href=&quot;http://golang.org/&quot;&gt;Go&lt;/a&gt; is one of my favorite programming languages.&amp;nbsp; The ethos of Go has historically been closer to that of C, but seems mostly to try to improve on the things that make C less than ideal for lots of projects.&lt;div&gt;
65. <br /></div>
66. <div>
67. One of the challenges that Go has always had is it's very weak support for versioning, dependency management, and vendoring.  The Go team's historic promise and premise (called the <a href="https://golang.org/doc/go1compat">Go1 Promise</a>) was that the latest version in any repo should always be preferred. This has a few ramifications:</div>
68. <div>
69. <br /></div>
70. <div>
71. <ul>
72. <li>No breaking changes permitted in a library, or package, ever.</li>
73. <li>The package should be "bug-free" at master.  (I.e. regression free.)</li>
74. <li>The package should live forever.</li>
75. </ul>
76. </div>
77. <div>
78. <br /></div>
79. <div>
80. For small projects, these are noble goals, but over time it's been well demonstrated that this doesn't work. APIs just too often need to evolve (perhaps to correct earlier mistakes) in incompatible ways. Sometimes its easier to discard an older API than to update it to support new directions.</div>
81. <div>
82. <br /></div>
83. <div>
84. Various 3rd party solutions, such as <a href="http://gopkg.in/">gopkg.in</a>, have been offered to deal with this, by providing some form of semantic versioning support.</div>
85. <div>
86. <br /></div>
87. <div>
88. Recently, go1.11 was released with an opt-in new feature called "<a href="https://github.com/golang/go/wiki/Modules">modules</a>".  The premise here is to provide a way for packages to manage dependencies, and to break away from the historic pain point of $GOPATH.</div>
89. <div>
90. <br /></div>
91. <div>
92. Unfortunately, with go modules, they have basically tossed the Go1 promise out the window. </div>
93. <div>
94. <br /></div>
95. <div>
96. Packages that have a v2 in their import URL (like my <a href="http://github.com/nanomsg/mangos-v2">mangos version 2</a> package) are assumed to have certain layouts, and are required to have a new <span style="font-family: Verdana, sans-serif;">go.mod</span> module file to be importable in any project using modules.  This is a new, unannounced requirement, and it broke my project from being used with any other code that wants to use modules.  (As of right now this is still broken.)</div>
97. <div>
98. <br /></div>
99. <div>
100. At the moment, I'm believing that there is no way to correct my repository so that it will be importable by both old code, and new code, using the same import URL.  The "magical" handling of "v2" in the import path seems to preclude this.  (I suspect that I probably need different contradictory lines in my HTML file that I use to pass "<span style="font-family: Verdana, sans-serif;">go-imports</span>", depending on whether someone is using the new style go modules, or the old style $GOPATH imports.)</div>
101. <div>
102. <br /></div>
103. <div>
104. The old way of looking at vendored code is no longer used.  (You can opt-in to it if you want still.)</div>
105. <div>
106. <br /></div>
107. <div>
108. It's entirely unclear how <span style="font-family: Verdana, sans-serif;">godoc</span> is meant to operate the presence of modules.  I was trying to setup a new repo for a v2 that might be module safe, but I have no idea how to direct godoc at a specific branch.  Google and go help doc were unhelpful in explaining this.</div>
109. <div>
110. <br /></div>
111. <div>
112. This is all rather frustrating, because getting away from $GOPATH seems to be such a good thing.</div>
113. <div>
114. <br /></div>
115. <div>
116. At any rate, it seems that go's modules are not yet fully baked.  I hope that they figure out a way for existing packages to automatically be supported without requiring reorganizing repos.  (I realize that this is already true for most packages, but for some -- like my mangos/v2 package -- that doesn't seem to hold true).</div>
117. <div>
118. <br /></div>
119. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/3737755355297793857/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=3737755355297793857' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/3737755355297793857'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/3737755355297793857'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/12/go-modules-so-much-promise-so-much.html' title='Go modules, so much promise, so much busted'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-2634434695144952901</id><published>2018-06-19T13:57:00.001-07:00</published><updated>2018-06-19T13:57:42.142-07:00</updated><title type='text'>Letter to Duncan Hunter (Immigration)</title><content type='html'>(Congressman Duncan Hunter is my Representative in the House.&amp;nbsp; Today I sent the letter below to him through the congressional email posting service (which verifies that I&#39;m his constituent).&amp;nbsp; A copy is here for others to read.&amp;nbsp; I encourage everyone, especially those in districts with Republican congressional representation to write a similar letter and send it to their congressman via the site at house.gov -- you can look up your own representative on the same site.)&lt;br /&gt;
120. <br />
121. Congressman Hunter,<br />
122. <br />
123. I have read your "position" statement with respect to the administrations "Zero Tolerance" treatment towards immigration, and the separation of families seeking asylum, and I am *most* dismayed by the position you have taken.<br />
124. <br />
125. I would encourage you to start by reading the full text of the following court order, which describes the reprehensible actions taken by the administration: https://t.co/adhGO6BDJR<br />
126. <br />
127. This is not hearsay, but legal findings by a US Court.<br />
128. <br />
129. Your claim that "most asylum seekers" are not being broken up is disturbing, because it also indicates that you believe it is okay to separate families "sometimes".  The reality is that there are cases where separation is warranted to protect the children, but there is ample evidence that this is not the case in many of these separations.  (See the above court case.)<br />
130. <br />
131. Not only that, but we know of cases where families have been separated for doing nothing wrong than presenting themselves fully legally at a border crossing and petitioning for asylum.<br />
132. <br />
133. Even in misdemeanor cases of illegal entry, the punishment of breaking families up -- while both children and parents are held separately for deportation proceedings, is both cruel and unusual punishment, and entirely unnecessary.<br />
134. <br />
135. It is also my belief that these separations create more risk to the United States, making us less safe.  Some of these children will remember the horrible ways that they were treated as criminals by our country.  How many will radicalize as a result later in life?  Just one is too many, and utterly unnecessary.<br />
136. <br />
137. Furthermore, the use of the misery of these families as some kind of political card to achieve ends is both morally reprehensible and of dubious value -- the border wall is a boondoggle and will have little effective value. The real problem of immigration is the attraction of millions of jobs -- jobs that should be filled *legally*.  (So mandatory eVerify, with criminal prosecution against *EMPLOYERS* who violate rather than against poor immigrants is the real fix to that problem.  You backed mandatory eVerify, an action which I applaud.)<br />
138. <br />
139. The ends -- a border wall -- do NOT justify the means, which have been amply and justifiably compared with the approach used by Nazi Germany in the 1930s, when dealing with Jewish families.<br />
140. <br />
141. As a nation we are embarrassed by our Internment of Japanese Americans during the same time frame, and the US government has been called to account for that in the past.  Yet even in the midst of world war our leaders did not stoop to separating parents from the children, or to use children as some kind of pawns in a larger political scheme.<br />
142. <br />
143. Indeed, these actions are more akin to those used by terrorists -- literally using "terror" (in this case fear of breaking a family up, which any parent knows is amongst the most terrible things to be contemplated) to achieve political ends.<br />
144. <br />
145. Please think long and hard about your decision to stand with Trump in this regard.  If you stand with him here -- as an agent of terror and tyranny, then I cannot in good conscience stand with you.<br />
146. <br />
147. You have a unique opportunity to break from the party lines, and demonstrate moral courage here -- an opportunity which if taken would certainly win back my support.   Or you can side with forces evil in supporting actions that numerous industry and business leaders have called "morally reprehensible" and "inhumane".<br />
148. <br />
149. The choice is yours to make.  For now.<br />
150. <div>
151. <br /></div>
152. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/2634434695144952901/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=2634434695144952901' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2634434695144952901'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2634434695144952901'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/06/letter-to-duncan-hunter-immigration.html' title='Letter to Duncan Hunter (Immigration)'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-3515984913819585807</id><published>2018-06-09T22:43:00.000-07:00</published><updated>2018-06-09T22:43:35.621-07:00</updated><title type='text'>Self Publishing Lessons</title><content type='html'>Over the past several weeks I&#39;ve learned far more than I ever wanted to about the self publishing process.&amp;nbsp; I&#39;m posting some of my findings here in the hopes that they may help others.&amp;nbsp;&lt;br /&gt;
153. <br />
154. <h2>
155. TLDR;</h2>
156. <br />
157. If you're going with eBooks, and you should, consider using an author website to sell it "early", and once your book is finished publish it with Kindle Direct Publishing and Smashwords.  Keep the author website / store up even after, so you can maximize returns.  Price your eBook between $2.99 and $9.99.<br />
158. <br />
159. If you're going to go with Print, start with Amazon Kindle Direct Publishing first, unless you're only needing a small run of books printed only in the USA (in which case TheBookPatch.com looks good).  Once you're book is really done, and you're ready to branch out to see it available internationally and from other bookstores, publish it with Ingram Spark.<br />
160. <br />
161. Get and use your own ISBNs (From MyIdentifiers.com -- buy 10 at a time), and make sure you opt out of Kindle Select!<br />
162. <br />
163. More details are below.<br />
164. <br />
165. <h2>
166. eBook Formats</h2>
167. <br />
168. Let's start with ebook formats first.  Be aware that I'm writing this from California, with no "nexus" elsewhere, and electronic goods (when they are purely electronic) are not taxable here.  That means I haven't worried much about accounting for VAT or Sales Tax, because I don't have to.<br />
169. <br />
170. <h3>
171. Leanpub</h3>
172. <br />
173. Leanpub is how I started out, but they've altered there terms several times.  Right now their royalties are not substantially less than anyone else (they used to be), and you can get an even higher return selling through your own store (such as Selz.com).  The one thing they have over everyone else is their Markua tools, and their focus on helping authors in the early stages -- you can "publish" before the book is complete on Leanpub.  I'm not sure how useful Markua is to other people -- I don't use it at all.  If you have a book in progress, this will let you sell copies early.  But, they do take a cut -- 80%. Frankly, their business model seems a bit iffy right now, and I wouldn't like to put too many eggs in that basket.  You won't need an ISBN at Leanpub.  They pay 80% royalties, allow free updates (to a limit most authors are unlikely to hit).  Leanpub has very limited reach, and doesn't distribute to anywhere else.<br />
174. <br />
175. <h3>
176. Author Website</h3>
177. <br />
178. The cheapest and most cost effective way to sell your ebooks is to open your own author store.  Sites like Selz.com will let you do this for free, and only charge reasonable transaction fees.  With this approach you can get about 95% of the book sales.  You can publish as soon as you want, send updates as often as you want, and don't need ISBNs or anything like that. On the downside, you have to do a little more work to set things up.  You'll also have limited reach, and pretty much look like a fly by night operation.  If you want to "pre-publish" before a work is complete, this is a way to do that, without paying that 20% to Leanpub.  You can <i>also</i> leave this store open, and point to it from your personal author pages, even after you are working with the larger distributers.<br />
179. <br />
180. <h3>
181. Ingram Spark</h3>
182. <br />
183. Ingram Spark's ebook distribution service gets broad reach through relationships with the various outlets.  You can use them to get to Apple, Kobo, even Amazon.  And yet I would <i>not</i> recommend doing this.  First off they charge to set up the book, typically $25.  Then if you want to make a revision to the book, it's another $25.  And then you're typically going to set it up so that you get only 45% of the royalties (or 40% if you really messed up and didn't opt out of the Amazon agreement.) . Furthermore, I found that their conversion of my ePub to Kindle format was inferior, leading to a poor reading experience on those devices.  (I have some complex layout, and custom fonts, as the book is technical in nature.)   I had much better luck generating my own .mobi format and working with Amazon directly.   Their service takes forever to get back to you -- I'm still waiting while I try to remove my eBook from their distribution.  In short, I would not use Ingram Spark for eBook.  You also will need an ISBN if you use Ingram Spark.<br />
184. <br />
185. <h3>
186. Amazon (Kindle Direct Publishing)</h3>
187. <br />
188. Using the Kindle Direct Publishing was pretty easy, and this let me provide a .mobi file that was optimized for Kindle, addressing issues caused by converting from ePub.  (To be fair most authors won't have these problems.)  If you want to reach Kindle readers (and you do!), you should just set up a KDP account.  One word though -- <i>don't opt-in to Kindle Select!!</i>  Amazon is great, for distributing to Amazon customers.  But you don't want to give away your exclusivity.    There is a weird set of rules about royalties with KDP though.  If you want to get their best 70% (which won't be in all markets, but the main ones) you need to set your List Price between $2.99 and $9.99, inclusive.  (Other values are used for other currencies.)  Deducted from your 70% rate is the cost to transfer the data to the user, which turns out to be pretty cheap -- less than a dollar typically.  (But if you're only selling a $2.99 book, make sure you keep the file sizes down, or this will hurt your rates.)  You can opt for a flat 35% royalty instead, which might make sense if your book is heavy on content, and its required if your book is outside the price points.  (This is why you never see ebooks listed for $11.99 or something like that on Amazon.)<br />
189. <br />
190. <h3>
191. Smashwords</h3>
192. <br />
193. I just set up my account with Smashwords, and I'm thrilled so far.  It looks like you'll get about 80% royalties through their own store, and 60% if your book is bought through one of their partners -- which includes just about everyone -- including Apple, GooglePlay, Kobo, etc.  This gets you pretty much everywhere, <i>except</i> Amazon.  But you did set up a KDP account already right?  They take the royalty, and you're done.  There is one fairly severe draw back to Smashwords -- they want you to upload your manuscript as a specially formatted Word document.  (They do have direct ePub though, which you can use if you want.  I did this because I don't have a Word version of my book, and it would be difficult to get one -- it was authored in Asciidoctor.)   You will need an ISBN to get into their expanded distribution program, of course.  They will offer to sell you one, but I recommend you not do that and use your own.  (Especially if you're uploading your own ePub.)<br />
194. <br />
195. <h3>
196. Direct Retailer Accounts</h3>
197. <br />
198. You can maximize royalties by setting up direct accounts with companies like Apple, Kobo,  and Barnes&Noble.  In my experience, it just isn't worth it.  Dealing with these all is a headache, and it takes forever.  Some, like Google Play Store, are almost impossible to get into.  As the list gets large, the percentage of your distribution that are covered here diminishes, consider whether that extra 10% royalty rate is worth the headache.  Some of these will need ISBNs, and the pricing and royalties will all vary of course.<br />
199. <br />
200. <h2>
201. Printed Books</h2>
202. <br />
203. If you've spent a lot of time making a great book, you probably want to see it in print format, right?  Nothing is quite the same to an author as being asked to sign a physical copy of professionally bound book of their own work.  Note that it takes some extra effort to set up a book for print -- you'll need to ensure that you have a press-ready PDF (I had to purchase a copy of Adobe Acrobat DC so that I could properly preflight my files), and setting up the cover can be a challenge if you're not a designer.<br />
204. <br />
205. Note that details such as print margins, paper weight, and hence cover sizes, can vary between the different printers.  Be prepared to spend a lot of time if you decide to go down this road, and to have to spend a lot of time for <i>each</i> printer you use.<br />
206. <br />
207. <h3>
208. TheBookPatch.com</h3>
209. <br />
210. After doing some research, I decided to give these guys a shot at printing my first version.  I was really impressed with the quality -- while the first printing of my book had a number of issues, none of them were the fault of TheBookPatch -- they were all on me.  The problem with these guys is that they are <i>tiny.  </i>Almost nobody has ever heard of them, and you won't get be getting this listed at places like Barnes&Noble.  Additionally, they are rather expensive, particularly if you want to send books to places overseas.  At one point I wanted to send one copy of my book to the Netherlands.  The shipping cost was going to be about $80.  Needless to say, my relationship with TheBookPatch came to an abrupt end.  (I'd still recommend giving these guys a shot if you're printing books for your own use here in the USA.)   One big advantage is that they were able to put together an attractive cover using their website cover designer, with no special skills.  You also don't need an ISBN to print through TheBookPatch.com.<br />
211. <br />
212. <h3>
213. Ingram Spark</h3>
214. <br />
215. Ingram Spark has the best rates internationally, and is reputed to have excellent print quality.  My book is available from them.  They charge $49 to set it up, and $25 for updates.  This is super annoying, so I wouldn't publish with them until and unless you know that you're ready and need international distribution or want to see your printed book available via Barnes&Noble or other retailers.  They're also <i>slow</i>.  I ordered 3 copies of my book a week ago, and they only confirmed that they are shipping them today.  If you're serious about selling printed books widely, I would definitely go with them.  But unless you anticipate the volume, I'd hold off.  You will need an ISBN as well.  With Ingram Spark, you set up your royalty rates which are usually 45% of <b>net</b>.   Typically this means you'll get something like a 20-25% actual royalty, depending on the book.<br />
216. <br />
217. <h3>
218. Amazon KDP</h3>
219. <br />
220. Now available for authors, you can use Amazon Print on Demand.  After setting up the layout, and doing the work to ensure the quality is good -- which can take some effort -- it's pretty easy.  Amazon will sell you an ISBN if you want one -- I'm not sure if they are required for print books or not.  (I already had one from my Ingram Spark journey.)  Amazon gives a much better royalty, of 60% of <b>net</b>, and their printing costs for small runs seem to be fairly inexpensive, as is shipping.  For example, my 430 page, 2 lb (7.5"x9.25" paperback) book cost about $6 to print, and about $10 to ship.  That means that as my list price is $49.95, I can expect to receive about $20.  Amazon will cut into their own margins to discount the book as well, to optimize the price.  Having said all that, I'm still waiting for my proof, which Amazon apologized for taking an extra day or two to print -- I should be getting it in a couple of days (I opted for the cheap shipping -- you can't use your Prime account to ship author proofs which are made available to you <i>at cost</i>).  Their paper is thicker than Ingram's and so I had to redesign the cover, and their margins are stricter (my page numbers fell outside their strict .5" margins), so I wound up having to re-do the whole layout.  It would have been better if I had started with Amazon first.<br />
221. <br />
222. There are other print-on-demand players, but I've heard enough complaints about print quality when using them, that I just avoided them.  After all, if you're bothering to put your book into print, you want the results to reflect all the effort you put into it.<br />
223. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/3515984913819585807/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=3515984913819585807' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/3515984913819585807'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/3515984913819585807'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/06/self-publishing-lessons.html' title='Self Publishing Lessons'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-4008668063095415098</id><published>2018-06-04T20:56:00.001-07:00</published><updated>2018-06-04T20:56:47.153-07:00</updated><title type='text'>Altering the deal... again....</title><content type='html'>(No, this is not about GitHub or Microsoft... lol.)&lt;br /&gt;
224. <br />
225. Back in March (just a few months ago), I signed up on <a href="http://leanpub.com/">Leanpub</a> to publish the <a href="http://staysail.tech/books/nng_reference/index.html">NNG Reference Manual</a>.  I was completely in the dark about how to go about self-publishing a book, and a community member pointed me at Leanpub.<br />
226. <br />
227. Leanpub charged $99 to set up, back in March, and offered a 90% (minus 50 cents) royalty rate.  On top of it they let me choose a price from free, or $0.99 to $99.  Buyers could choose within that range.  This looked great, although I was a bit hesitant to spend the $99 since there was no way to try their platform out.<br />
228. <br />
229. Note that at this time I was not interested (and am still not interested) in their authoring tools based on Markua.  I had excellent tooling already in <a href="http://asciidoctor.org/">Asciidoctor</a>, plus a bunch of home-grown tools (that I've since further expanded upon) to markup and layout the book, plus previewing, etc.<br />
230. <br />
231. Everything was great, and I made sales ranging from $0.99 to $20.  Not a lot of sales, but enough to nearly recoup my $99 investment.  Now, I wasn't looking at this as a money making venture, but as a way to help support my work around NNG -- having a professionally produced reference manual was something I considered an important step for NNG.<br />
232. <br />
233. Shortly after I created the book and published, Leanpub changed the minimum price that buyers could pay to $4.99.  We're talking about a digital good here.  First time the deal was altered....<br />
234. <br />
235. Then in April, they introduced a new SaaS pricing model, where I could have ditched the $99 fee.  So I'm feeling like a chump, but hey at least I have that 90% royalty rate, right?  (By this time I'd sold enough to cover that initial $99 outlay, thanks to generous supporters from the NNG community.) . Deal altered <i>again</i>.<br />
236. <br />
237. Then they introduced a freemium model in May, where I really could have skipped that $99 outlay.  But they told me that I was grandfathered, so I could keep my 90% rate, so I was getting <i>something</i> for that $99 I spent originally.  Deal altered third time?<br />
238. <br />
239. Now, they've told me that they've changed their mind, and no, they aren't going to let me keep that grandfathered rate.  Deal altered <b>again?!?</b><br />
240. <br />
241. They posted a <a href="https://leanpub.com/grandfathering">long essay</a> explaining why they "had" to do this.  I get it, their old business model wasn't working.  But in the past 3 months they've made not one, not two, but <i>three</i> changes to their pricing and business model.  They've made promises, and gone back on their word.<br />
242. <br />
243. But it's ok, because at 80% I'm making more than with Amazon, right?  Well, no, not really.  I won't repeat the calculations here, but it turns out that I would have made slightly more money with Amazon.  Now, that's partly due to the fact that my sales have been quite slow (as they were predicted to be -- this is a really niche book -- a reference manual for a product that isn't even 1.0 yet.)<br />
244. <br />
245. The thing is, I'm slightly irked about the loss of income, but I'm much more angry about the lack of respect they've given us, their authors and customers.  Clearly, their promises don't carry much weight.  They've offered lifetime free Pro accounts to customers who were with them long enough to have at least $500 in royalties, but everyone else is out of luck.  As to those lifetime pro accounts -- well, it's "lifetime, or until we change our mind".   Which seems to occur about once a month.<br />
246. <br />
247. Now Leanpub isn't some big bad company, but their attitude and thinking reflected in how they've handled this process shows clear alignment with the same thought processes that those big bad companies have.  As an author you're not a valued partner to them -- you're a source of revenue, with very little effort on their part required to support you.<br />
248. <br />
249. I've started rethinking my use of Leanpub obviously.<br />
250. <br />
251. It seems like I can make use of <a href="https://selz.com/">Selz</a> which seems to have really good support for selling digital goods like eBooks (and even has a Pay What You Want option!), and with my small number of digital goods will only charge me the transaction processing costs -- either 2.9% or 3.9% depending on location.  (Digital goods are not taxable in California.)  So what was I gaining from Leanpub again?<br />
252. <br />
253. For Kindle and iBooks, it also looks like dealing with Amazon and Apple directly look like a better deal than Leanpub.  You get their expanded distribution, and yes, you only get 70% royalties, but you don't have to pay any recurring fees.  Unless you're doing large volumes, the math on these works out better than any of the Leanpub paid plans.<br />
254. <br />
255.  (IngramSpark, where I have also posted the book, also works, but I've had less than satisfactory results with their epub->mobi conversion, so I can't recommend using them for Kindle at least, and I think the royalties you get from dealing directly with Apple are superior anyway.)<br />
256. <br />
257. This all seems like a lot of work, but I hope this helps other authors who might be considering using Leanpub.<br />
258. <br />
259. (There is one feature which is nice on Leanpub, which is the ability to publish an incomplete work in progress, and then keep updating it.  But let's face it, you can do that equally well from your own website and something like Selz.)<br />
260. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/4008668063095415098/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=4008668063095415098' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4008668063095415098'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4008668063095415098'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/06/altering-deal-again.html' title='Altering the deal... again....'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-5545029275759344419</id><published>2018-06-04T15:46:00.000-07:00</published><updated>2018-06-04T15:46:12.130-07:00</updated><title type='text'>Not Abandoning GitHub *yet*</title><content type='html'>The developer crowds are swarming off of GitHub in the wake of today&#39;s announcement that Microsoft has agreed to purchase GH for $7.5B.&lt;div&gt;
261. <br /></div>
262. <div>
263. I've already written why I think this acquisition is good for neither GitHub nor Microsoft.  I don't think it's good for anyone else either... but maybe at least it alerts us all to the dangers of having all our eggs in the same basket.</div>
264. <div>
265. <br /></div>
266. <div>
267. At the moment my repositories will <i>not</i> be moving.  The reason for this is quite simple -- while the masses race off of GitHub, desperate for another safe harbor, the panic that this has created is overwhelming alternative providers.  GitLab reported a 10X growth.  While this might be good for GitLab, its not good for people already on GitLab, as there was already quite a well understand performance concern around GitLab.com.</div>
268. <div>
269. <br /></div>
270. <div>
271. At least in the short term, GitHub's load will decrease (at least once all the code repo exports are done), I think. </div>
272. <div>
273. <br /></div>
274. <div>
275. The other thing is that Microsoft has come out and made some pretty strong promises about not altering the GitHub premise, and the "new leadership" over there is ostensibly quite different from the old.  (Having said that, there is a <b>lot</b> of bad blood and history between FOSS and Microsoft. A lot of the current generation of millenials don't have that history, but some of us haven't forgotten when Steve Ballmer famously said "Linux is a cancer", and when Microsoft used every dirty trick in the book to try to kill all competitors, including open source software.  If Microsoft had had its way back in the 90s and 00s, the Internet would have been a company shanty-town, and Linus Torvalds would have been a refugee outlaw.</div>
276. <div>
277. <br /></div>
278. <div>
279. Thankfully that didn't happen.</div>
280. <div>
281. <br /></div>
282. <div>
283. Microsoft is trying to clean its image up, and maybe it is reformed now, but the thing we all have to remember is that Microsoft is beholden first, foremost, and exclusively to it's shareholders.  Rehabiliting it's image is critical to business success today, but at it's roots Microsoft still has those same obligations.)</div>
284. <div>
285. <br /></div>
286. <div>
287. The past couple of years of good behavior doesn't undo decades of rottenness; many of us would have been thrilled to see Microsoft enter chapter 11 as the just dessert for its prior actions.</div>
288. <div>
289. <br /></div>
290. <div>
291. Microsoft was losing mindshare to OSS and software like Git (and companies like GitHub). Purchasing GitHub is clearly an effort to become relevant again.   The real proof will be seen if Microsoft and GitHub are still as FOSS friendly in two years as they are today.  Promises made today are cheap.</div>
292. <div>
293. <br /></div>
294. <div>
295. But I'm willing to let them have the benefit of the doubt, understanding that I retain my options to depart at any time.  I won't be creating *new* repositories there, and my private one's will be moving off of GitHub because I don't want Microsoft to have access to my proprietary work.  (Probably they can still get it from backups at GitHub, but we do what we can...)</div>
296. <div>
297. <br /></div>
298. <div>
299. But my open source stuff is still there.  For now.</div>
300. <div>
301. <br /></div>
302. <div>
303. That means mangos, NNG, nanomsg, and tcell remain.  For now.</div>
304. <div>
305. <br /></div>
306. <div>
307. It's up to Microsoft and GitHub to see if they stay.</div>
308. <div>
309. <br /></div>
310. <div>
311.  - Garrett</div>
312. <div>
313. <br /></div>
314. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/5545029275759344419/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=5545029275759344419' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/5545029275759344419'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/5545029275759344419'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/06/not-abandoning-github-yet.html' title='Not Abandoning GitHub *yet*'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-6277133528282402365</id><published>2018-06-03T18:57:00.000-07:00</published><updated>2018-06-03T18:57:04.175-07:00</updated><title type='text'>Microsoft Buying GitHub Would be Bad</title><content type='html'>So apparently &lt;a href=&quot;https://www.bloomberg.com/news/articles/2018-06-03/microsoft-is-said-to-have-agreed-to-acquire-coding-site-github&quot;&gt;Microsoft wants to buy GitHub&lt;/a&gt;.&lt;br /&gt;
315. <br />
316. This is a <i>huge</i> mistake for both companies, and would be tragic for pretty much everyone involved.<br />
317. <br />
318. GitHub has become <i>the</i> open source hosting site for code, and for a number of companies, it also hosts private repositories.  It's the place to be if you want your code to be found and used by other developers, and frankly, its so much of a <i>de facto</i> standard for this purpose that many tools and services work better with GitHub.<br />
319. <br />
320. GitHub was founded on the back of Git, which was invented by Linus Torvalds to solve source code management woes for the Linux kernel. (Previously the kernel used an excellent tool called BitKeeper for this job, but some missteps by the owners of BitKeeper drove the Linux team away from it.  It looks like GitHub is making similar, albeit different, commercial missteps.)<br />
321. <br />
322. Microsoft already has their own product, <a href="https://www.visualstudio.com/team-services/">Visual Studio Team Services</a>, which competes with GitHub, but which frankly appeals mostly to Microsoft's own developer base.  I don't think it is widely used by Linux developers for example.<br />
323. <h2>
324. <br /></h2>
325. <h2>
326. Implications for Open Source</h2>
327. <br />
328. Microsoft has been much more "open source friendly" of late, but I have to admit I still don't trust them.  I'm hardly alone in this.<br />
329. <br />
330. It also is a breach of sorts of an unwritten trust that the open source community has placed in them.  There is much bad blood between Microsoft and open source software.  Many of the most treasured open source systems exist directly in conflict to proprietary systems.  Think about software like Samba, and Wine and OpenOffice.  These were created as <i>alternatives</i> to Microsoft.  Being acquired by Microsoft means that these projects will feel compelled to abandon GitHub.<br />
331. <br />
332. As this happens, many tools and services that offer services that are tailored to GitHub (automated code review, CI/CD, etc.) are going to be rushing to find a way to offer services for alternatives, as their client base runs screaming from GitHub.  (Back in February of 2016 I <a href="http://garrett.damore.org/2016/02/leaving-github.html">tried to leave GitHub</a>, because of philosophical differences of opinion with their leadership.  I abandoned the effort after discovering that too many of the external support services I used for these open source projects were either GitHub only, or could only be converted away from GitHub with large amounts of additional effort and big negative impact for my users.)<br />
333. <br />
334. This is a watershed moment for GitHub.<br />
335. I predict in as little as 6 months nobody will be creating new open source projects on GitHub.<br />
336. <br />
337. Unfortunately, it's probably already too late for GitHub.  Unless they were to come out and immediately deny any acquisition attempts, and make some public announcements recognizing the trust they've been given, and asserting the importance of honoring it, nobody will trust them any more.<br />
338. <h2>
339. <br /></h2>
340. <h2>
341. Implications for Commercial Use</h2>
342. <br />
343. This is also going to harm commercial customers, driving them away.<br />
344. <br />
345. Microsoft has many commercial ventures which overlap with those of almost everyone doing anything in software.  GitHub being acquired by Microsoft will in one fell swoop make GitHub a direct <i>competitor</i> with vast amounts of their own customer base.  (Essentially, your either a Microsoft competitor, or a partner.  And often both.)<br />
346. <div>
347. <br /></div>
348. If you're using GitHub for private repositories, it probably is time to rethink that.  Unless you trust Microsoft not to do evil.  (They've never even made any such promises.)  This also means, I think that it might be time to reconsider hosting your private data with <i>anyone</i> else.  GitLab and BitBucket look better to be sure, but what's to prevent another large company from acquiring them?<br />
349. <br />
350. It's time to reconsider the cost of hosting in the cloud.  I've been expecting a move back to on-premises storage and hosting for some time now, but this will only accelerate that.<br />
351. <h2>
352. <br /></h2>
353. <h2>
354. Implications for Microsoft</h2>
355. <br />
356. Microsoft will spend quite a lot of money to acquire GitHub.  But instead of acquiring a goose that lays golden eggs, they are going to have one that needs to be fed and turns that into fecal material.<br />
357. <br />
358. At the same time, while this may help bolster some of the technology in VSTS in the short term, the reality is that most of the best stuff isn't that hard to build, and most of what GitHub has can be done on any cloud based system with sufficient storage and compute.  Most of their tech is not tied to Windows, almost certainly.<br />
359. <br />
360. The VSTS team will no doubt be impacted, and there will be a lot of pain and suffering attempting to more tightly integrate VSTS with the new adopted child.  I'm sure there are redundancies that will be eliminated, but I expect part of what is going to happen is a shift in focus from providing the best experience for Visual Studio developers and making things work well on Azure, to figuring out how to more tightly integrate GitHub's toolset into theirs.  Can you imagine trying to reconcile the differences between VSTS and GitHub's issue tracking systems?  Yikes!<br />
361. <br />
362. The uncertainty will annoy customers, and I suspect will drive them away from the existing VSTS stack.  Whey they leave, they probably won't be moving to GitHub.<br />
363. <br />
364. Like the proverbial dog with the bone looking at his reflection in the water while on the bridge, instead of having one bone, Microsoft's greed will leave it with none (at least in this space.)<br />
365. <br />
366. I'm sure that the founders and investors of GitHub will make a mint taking Microsoft's money.  Normally I'd applaud anyone with plans to part Microsoft from some of it's funds.  But this move is just plain bad business.<br />
367. <h2>
368. <br /></h2>
369. <h2>
370. Anti-Trust Violations?</h2>
371. <br />
372. As I mentioned above, Microsoft has their own product, Visual Studio Team Services, which competes with GitHub.  This alleged acquisition of GitHub seems to me to fly in the face of anti-trust rules.  Microsoft clearly has been trying to make inroads into the open source community with projects like Visual Studio Code and Linux support for VSTS, so I would hope that the regulatory bodies involved would examine this with great scrutiny.<br />
373. <br />
374. Of course, if GitHub is for sale, many of the same concerns <i>except</i> the antitrust legislation would apply.  It would be a Bad Thing (tm) if GitHub were to be acquired by Facebook, Google, or Amazon, for example, for most of the same reasons that being acquired by Microsoft would be bad.<br />
375. <br />
376. Now please pardon me while I go back to setting up <a href="https://gogs.io/">gogs</a> on my own systems...<br />
377. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/6277133528282402365/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=6277133528282402365' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/6277133528282402365'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/6277133528282402365'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/06/microsoft-buying-github-would-be-bad.html' title='Microsoft Buying GitHub Would be Bad'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-7480280191192932403</id><published>2018-05-22T11:06:00.001-07:00</published><updated>2018-05-22T13:04:51.722-07:00</updated><title type='text'>No, Nanomsg is NOT dead</title><content type='html'>There seems to have been some pretty misleading data out on the Internet, indicating that &quot;&lt;a href=&quot;http://nanomsg.org/&quot;&gt;nanomsg&lt;/a&gt; is dead&quot;.&amp;nbsp; The main culprit here is a &quot;&lt;a href=&quot;http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/&quot;&gt;postmortem&lt;/a&gt;&quot; by Drew Crawford.&amp;nbsp; Unfortunately comments are apparently not working on that post according to Drew himself.&lt;br /&gt;
378. <br />
379. The thing is this (apologies to Samuel Clemens):  "reports of the death of nanomsg have been greatly exaggerated".<br />
380. <br />
381. So it's time to set the record straight.<br />
382. <br />
383. I've been working hard on nanomsg, and the Scalability Protocols that are intrinsic to nanomsg for quite some time.  It has been generally occupied my full time paid job for approximately the past year.  I've been working on this stuff part time for longer than that.<br />
384. <br />
385. The main focus during this time has been a complete rewrite of the core library, known as <a href="http://github.com/nanomsg/nng">NNG</a>.  NNG, or nanomsg-next-gen, aims to be a far superior version of nanomsg, with significant new capabilities, greatly improved reliability, scalability, extensibility, and maintainability.  It is wire compatible with legacy nanomsg and <a href="http://github.com/go-mangos/mangos">mangos</a>, and retains a backwards compatible API (though it also offers a newer API which should be quite a lot easier to use).<br />
386. <br />
387. During all this time, I've continued to act as the maintainer for nanomsg, although at this point I'd say that nanomsg itself is in sustaining mode, as I'm very focused on having NNG stand in as a full replacement for nanomsg.<br />
388. <br />
389. We've also published the first <a href="https://leanpub.com/nngmanual">NNG book</a>, which is really just the reference manual.  There are over 400 pages (actually about 650 in the 7.5"x9.25" printed edition, which I've not put up yet) of detailed API documentation available.  (Let me know if you're interested in the print edition -- it costs me about $35 to produce, but I'm willing to make it available for folks that are willing to pay for it.  Admittedly the electronic version is probably a lot more useful since it has working hyperlinks and supports searching.)  Oh, and by the way, the book also covers the legacy API used with legacy libnanomsg.<br />
390. <br />
391. I'm working on the second NNG book, which will be much more of a "how-to" (backed up with case studies and code) now.  (This will take some time to come to market.  At the moment these books are a secondary effort, since the time spent on them is time spent away from working on the code itself or on related commercial activities.)<br />
392. <br />
393. There have been more contributors to NNG of late, and interest is picking up as NNG itself is already on final countdown for its FCS approach.  (The first beta release, 1.0.0-beta.1 was released last week.  I expect to release a 2nd beta today, and then the final release will probably come a week or so later, depending upon beta test results of course.)<br />
394. <br />
395. The work I've done for NNG has also inspired me to make further improvements to mangos.  Over the course of the next few months you can expect to see further harmonization between these two projects as NNG gains support for the STAR protocol from mangos, and mangos gains some new capabilities (such as optional separable contexts to enable much easier development of concurrent applications.)<br />
396. <br />
397. So, if you've heard that "nanomsg is dead", now you know better.  In fact, I'd venture to say that the project is healthier and more alive than it ever was.<br />
398. <br />
399. Furthermore, in many respects the new NNG project is far more robust, scalable, and stable than I believe nanomsg or ZeroMQ have <i>ever</i> been.  (This because NNG has been designed with a serious eye towards production readiness from the first line of code.  Every error case is carefully considered.)<br />
400. <br />
401. If you haven't looked at any this stuff lately, give it another look!<br />
402. <br />
403. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/7480280191192932403/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=7480280191192932403' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/7480280191192932403'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/7480280191192932403'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/05/no-nanomsg-is-not-dead.html' title='No, Nanomsg is NOT dead'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-5739691938382858634</id><published>2018-01-23T09:50:00.004-08:00</published><updated>2018-01-23T09:50:46.925-08:00</updated><title type='text'>Why I&#39;m Boycotting Crypto Currencies</title><content type='html'>Unless you&#39;ve been living under a rock somewhere, you probably have heard about the crypto currency called &quot;Bitcoin&quot;.&amp;nbsp; Lately its skyrocketed in &quot;value&quot;, and a number of other currencies based on similar mathematics have also arisen.&amp;nbsp; Collectively, these are termed cryptocurrencies.&lt;br /&gt;
404. <br />
405. The idea behind them is fairly ingenious, and based upon the idea that by solving "hard" problems (in terms of mathematics), the currency can limit how many "coins" are introduced into the economy.  Both the math and the social experiment behind them is something that on paper looks really interesting.<br />
406. <br />
407. The problem is that the explosion of value has created a number of problems, and as a result I won't be accepting any of these forms of currencies for the foreseeable future.<br />
408. <br />
409. First, the market for each of these currencies is controlled by a relatively small number of individuals who own a majority of the outstanding "coins".  The problem with this is that by collusion, these individuals can generate "fake" transactions, which appear to drive up demand on the coins, and thus lead to a higher "value" (in terms of what people might be willing to pay).  The problem is that this is a "bubble", and the bottom will fall right out if enough people try to sell their coins for hard currency.  As a result, I believe that the value of the coins is completely artificial, and while a few people might convert some of these coins into hard cash for a nice profit, the majority of coin holders are going to be left out in the cold.<br />
410. <br />
411. Second, the "cost" of performing transactions for some of these currencies is becoming prohibitively expensive.  With most transactions of real currency, its just a matter of giving someone paper currency, or running an electronic transaction that normally completes in milliseconds.  Because of the math associated with cryptocurrencies, the work to sign block chains becomes prohibitive, such that for some currencies transactions can take a lot of time -- and processors are now nearly obliged to charge what would be extortionary rates just to cover their own costs (in terms of electricity and processing power used).<br />
412. <br />
413. The environmental impact, and monumental waste, caused by cryptocurrencies cannot be overstated.  We now have huge farms of machines running, consuming vast amounts of power, performing no useful work except to "mine" coins.  As time goes on, the amount of work needed to mine each coin grows significantly (an intentional aspect of the coin), but what this means is that we are burning large amounts of power (much of which is fossil-fuel generated!) to perform work that has no useful practical purpose.   Some might say something similar about mining precious metals or gems, but their a many many real practical applications for metals like gold, silver, and platinum, and gems like diamonds and rubies as well.<br />
414. <br />
415. Finally, as anyone who wants to build a new PC probably realizes, the use of computing hardware, and specifically "GPUs" (graphical processing units, but which also can be used to solve many numerical problems in parallel) have increased in cost dramatically -- consumer grade GPUs are generally only available today for about 2x-3x their MSRPs.  This is because the "miners" of cryptocurrencies have snapped up every available GPU.  The upshot of this is that the supply of this hardware has become prohibitive for hobbyists and professionals alike.  Indeed, much of this hardware would be far far better used in HPC arenas where it could be used to solve real-world problems, like genomic research towards finding a cure for cancer, or protein folding, or any number of other interesting and useful problems which solving would benefit mankind as a whole.  It would not surprise me if a number of new HPC projects have been canceled or put on hold simply because the supply of suitable GPU hardware has been exhausted, and putting some of those projects out of budget reach.<br />
416. <br />
417. Eventually, when the bottom does fall out of those cryptocurrencies, all that GPU hardware will probably wind up filling land-fills, as many people won't want to buy used GPUs, which may (or may not) have had their lifespans shortened.  (One hopes that at least the eWaste these cause will be recycled, but we know that much eWaste winds up in landfills in third world countries.)<br />
418. <br />
419. Crypto-curency mining is probably one of the most self-serving and irresponsible (to humanity and our environment) activities one can take today, while still staying in the confines of the law (except in a few jurisdictions which have sensibly outlawed cryptocurrencies.)<br />
420. <br />
421. It&#39;s my firm belief that the world would be far better off if crypto-currencies had never been invented.</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/5739691938382858634/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=5739691938382858634' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/5739691938382858634'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/5739691938382858634'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2018/01/why-im-boycotting-crypto-currencies.html' title='Why I&#39;m Boycotting Crypto Currencies'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-6814406495840619395</id><published>2017-11-22T17:32:00.003-08:00</published><updated>2017-11-30T10:42:12.606-08:00</updated><title type='text'>Small Business Accounting Software Woes</title><content type='html'>I&#39;m so disappointed with the online accounting software options available to me; and I&#39;ve spent &lt;i&gt;far &lt;b&gt;fa&lt;/b&gt;&lt;/i&gt;r too much time in the past couple of days looking for an accounting solution for my new business.  The current state of affairs makes me wonder if just using a spreadsheet might be as easy.&lt;br /&gt;
422. <br />
423. I am posting my experiences here for two reasons.<br />
424. <ol>
425. <li>To inform others who might have similar needs, and</li>
426. <li>To inform the hopefully smart people at these companies, so maybe they will improve their products.</li>
427. </ol>
428. Let me start with a brief summary of my needs:<br />
429. <br />
430. <ul>
431. <li>Track time (esp. billable hours)</li>
432. <li>Tracked time should include date, and project/client, and some description of work performed.</li>
433. <li>Multiple currency support.  I have international clients that I need to bill in their preferred currency.</li>
434. <li>Invoicing and payment tracking for above.</li>
435. <li>Payroll -- preferably integrated with someone like <a href="https://gusto.com/">Gusto</a>.</li>
436. <li>Support for two employees with plans to grow. </li>
437. <li>Double-entry accounting (including bank reconciliation) for my accountant.</li>
438. <li>Affordable -- I'm a <i>small</i> business owner.</li>
439. </ul>
440. That's it.  Nothing super difficult, right?  You'd think there would be dozens of contenders who could help me.<br />
441. <br />
442. You'd be wrong.<br />
443. <br />
444. Here's what I looked at, and their deficiencies:<br />
445. <br />
446. <h4>
447. <a href="http://www.freshbooks.com/">Freshbooks</a> </h4>
448. <a href="https://www.bing.com/th?id=A4703f29460ecbae785b6b801e5b59d18&w=110&h=110&c=7&rs=1&qlt=95&pcl=f9f9f9&cdv=1&pid=16.1" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="110" data-original-width="110" src="https://www.bing.com/th?id=A4703f29460ecbae785b6b801e5b59d18&w=110&h=110&c=7&rs=1&qlt=95&pcl=f9f9f9&cdv=1&pid=16.1" /></a><br />
449. <br />
450. I really like most of what Freshbooks has to offer, and this was my starting point.  Super easy to use, an integration with <a href="https://gusto.com/">Gusto</a>, and their invoicing solution is super elegant.  Unfortunately, their lack of reconciliation and double-entry accounting (or any of the other "real" accounting stuff) disqualifies them.  Adding to the problem, I already use them for my personal consulting business (where I've been a happy user), and they don't have support for multiple business on their "Classic Edition".<br />
451. <br />
452. Then there is the whole confusion between "New Freshbooks" and "Classic Freshbooks".<br />
453. <br />
454. This is a company that states they intend to continue to keep two separate software stacks, with about 90% functionality overlap, running ~forever.  Why? Because they have some features (and some integrations) that they lack in the new one. (I've been informed that my use patterns indicate that I should stay on the "Classic" edition forever because of my heavy use of Time Tracking.)
455.  
456. Some of us with real world software engineering experience know how costly and hateful it is to have multiple simultaneous versions of a product in production.  Freshbook's approach here, with no plans to merge the functionality, is about the most boneheaded decision I've seen engineering management take.<br />
457. <br />
458. Being stuck on the "Classic Edition" makes me feel like a loser, but really it's a sign that their own product is the loser.  I have to believe at some point one product or the other is going to be a dead end.<br />
459. <br />
460. <h4>
461. <a href="https://quickbooks.intuit.com/online/">Quickbooks Online</a><div class="separator" style="clear: both; text-align: center;">
462. <a href="https://quickbooks.intuit.com/online/"></a><a href="https://www.bing.com/th?id=A7dd06b14cf008dc8cc3c275777455440&w=110&h=110&c=7&rs=1&qlt=80&pcl=f9f9f9&cdv=1&pid=16.1" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="110" data-original-width="110" src="https://www.bing.com/th?id=A7dd06b14cf008dc8cc3c275777455440&w=110&h=110&c=7&rs=1&qlt=80&pcl=f9f9f9&cdv=1&pid=16.1" /></a></div>
463. </h4>
464. <br />
465. This is a product that is well recommended, and probably one of the most widely used.  It has so much capability.  It also lacks the "hacked together by a bunch of different engineering teams that didn't talk to each other" feeling that their desktop product has.  (Yes, I have experience with Quickbooks Pro, too.  Sad to say.)  It's probably a good thing I can't look at their code behind the curtain.<br />
466. <br />
467. The biggest, maybe even only, failing they have for my use case is their inability to bill against clients that are in a different currency.  Wait, they are multicurrency capable, right?  Uh, no they aren't.  If I can't record my billable hours against a client in another country in their preferred currency, then whatever you think your "multicurrency" support is doesn't count. I have international clients that demand billing in their local currency.  So this is a non-starter for me.  This feature has been asked for before from them, and they have ignored it.  Major, and honestly unexpected, fail.<br />
468. <br />
469. Cost wise they aren't the cheapest, but this one feature absence is a show stopper for me, otherwise I'd probably have settled here.<br />
470. <br />
471. <h4>
472. <a href="https://www.xero.com/">Xero</a></h4>
473. <div class="separator" style="clear: both; text-align: center;">
474. <a href="https://www.bing.com/th?id=Abbb615bc6471c8c98a9107d4aed0a199&w=110&h=110&c=7&rs=1&qlt=95&pcl=f9f9f9&cdv=1&pid=16.1" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="110" data-original-width="110" src="https://www.bing.com/th?id=Abbb615bc6471c8c98a9107d4aed0a199&w=110&h=110&c=7&rs=1&qlt=95&pcl=f9f9f9&cdv=1&pid=16.1" /></a></div>
475. <br />
476. Xero is another of the main companies, and in Gartner's magic quadrant as their leader in the sector.  I didn't actually try them out -- though I did research.  Their shortcomings for me were: price (multi-currency support requires me to pay $70 / month, which is about 2x all the others), and lack of time tracking.  Sure, I can add an integration from some other company like Tsheets, for another $20 / month.  But now this solution is like 3x the cost of everyone else.<br />
477. <br />
478. One feature that Xero includes for that $70 is payroll processing -- but only for a handful of states (California is one), and I can't seem to find any reviews for folks who have used them.   If I want to use an outside company with a longer track record and broader coverage across states, like SurePayroll or Gusto or ADP, I will wind up paying double.<br />
479. <br />
480. If Xero would change their menu somewhat (make it ala carte), we'd be able to work together.  Let me integrate with Gusto, and not have to pay exorbitant fees for multi-currency support.  Add time tracking and it would be even better.<br />
481. <br />
482. Arguably I could stop being such a penny pincher, and just go with Xero + Tsheets or somesuch.  Outside of the crazy expensive options for companies that can afford a full time accountant (Sage, NetSuite, looking at you!), this was the most expensive option.  I'd also have to use Xero's payroll service, and I'm not sure<br />
483. <h4>
484. </h4>
485. <h4>
486. <a href="https://zipbooks.com/">ZipBooks</a></h4>
487. <br />
488. <a href="https://tse1.mm.bing.net/th?id=OIP.pYz5QsJGR3hh1YGXDMUhkAEyDM&w=230&h=160&c=7&qlt=90&o=4&pid=1.7" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="160" data-original-width="230" height="139" src="https://tse1.mm.bing.net/th?id=OIP.pYz5QsJGR3hh1YGXDMUhkAEyDM&w=230&h=160&c=7&qlt=90&o=4&pid=1.7" width="200" /></a>At first blush, ZipBooks looked like a great option.  On paper they have everything I need -- they even partnered with Gusto, and claim to have multicurrency support.  Amazingly, they are even <b>free</b><i>.  </i>Of course if you elect to use some of their add-ons, you pay a modest fee, but from a pure price perspective, this looks like the cheapest.<br />
489. <br />
490. Unfortunately, as I played with their system, I found a few major issues.  Their multi-currency support is a bit of an inconvenient joke. They don't let you set a per-client currency.  Instead you change the currency for the entire account, then generate invoices in that currency (or accept payments), then have to switch back to the home currency.  This is account wide, so you better not have more than one person access the account at a time.  The whole setup feels really hinky, and to be honest I just don't trust it.<br />
491. <br />
492. Second, their bank integration is (as of today) broken -- meaning the website gives me conflict errors before I even can select a bank (I wanted to see if my business bank -- a regional smaller bank -- is on their list).  So, not very reliable.<br />
493. <br />
494. Finally, their support is nearly non-existent.  I sent several questions to them through their on-line support channel, and got back a message "ZipBooks usually responds in a day".  A day.  Other companies I looked at took maybe 10-20 minutes to respond -- I still have not received a response from ZipBooks.<br />
495. <br />
496. I need a service that supports real multicurrency invoicing, is reliable, and with reachable support. Three strikes for ZipBooks.  Damn, I really wanted to like these guys.<br />
497. <br />
498. <h4>
499. <a href="https://www.kashoo.com/home">Kashoo</a></h4>
500. <div class="separator" style="clear: both; text-align: center;">
501. <a href="https://www.kashoo.com/hs-fs/hubfs/kashoo_logo_BR_2.png?t=1511390404361&width=225&name=kashoo_logo_BR_2.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="65" data-original-width="225" height="57" src="https://www.kashoo.com/hs-fs/hubfs/kashoo_logo_BR_2.png?t=1511390404361&width=225&name=kashoo_logo_BR_2.png" width="200" /></a></div>
502. <br />
503. Kashoo was well reviewed, but I had some problems with them.  First their only payroll integration is with SurePayroll.  I hate being locked in, although I could probably overlook this.  Second, they don't have any time tracking support. Instead they partner with Freshbooks, but only the "Classic Edition" (and apparently no plans to support the "New Freshbooks".)  A red flag.<br />
504. <br />
505. And, that brings in the Freshbooks liability (only one company, so I can't have both my old consulting business and this new one on the same iOS device for example), and I'd have to pay for Freshbooks service too.<br />
506. <br />
507. On the plus side, the Kashoo tech support (or pre-sales support?) was quite responsive.  I don't think they are far off the mark.<br />
508. <br />
509. <h4>
510. <a href="https://tse1.mm.bing.net/th?id=OIP.Z5dPEloKGAzqx_c79tp9rgEsCo&w=188&h=105&c=8&rs=1&qlt=90&pid=3.1&rm=2" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="105" data-original-width="188" height="110" src="https://tse1.mm.bing.net/th?id=OIP.Z5dPEloKGAzqx_c79tp9rgEsCo&w=188&h=105&c=8&rs=1&qlt=90&pid=3.1&rm=2" width="200" /></a><a href="https://www.waveapps.com/">Wave Accounting</a> </h4>
511. <br />
512. Wave is another free option, but they offer payroll (although full service only in five states) as an add-on.  (They also make money on payment processing, if you use that.)  Unfortunately, they lacked support for integrations, time tracking, or multi-currency support.  I'd like to say close but no cigar, but really in this case, it's just "no cigar".  (I guess you get what you pay for...)<br />
513. <br />
514. <h4>
515. <a href="http://www.zoho.com/books">Zoho Books</a><div class="separator" style="clear: both; text-align: center;">
516. <a href="http://www.zoho.com/books"></a><a href="https://tse1.mm.bing.net/th?id=OIP.iNwlihSOU5uUVMVUxAEvFQD0CC&w=190&h=105&c=8&rs=1&qlt=90&pid=3.1&rm=2" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="105" data-original-width="190" height="110" src="https://tse1.mm.bing.net/th?id=OIP.iNwlihSOU5uUVMVUxAEvFQD0CC&w=190&h=105&c=8&rs=1&qlt=90&pid=3.1&rm=2" width="200" /></a></div>
517. </h4>
518. <br />
519. Zoho Books is another strong option, well regarded.  So far, it seems to have everything I need <i>except</i> any kind of payroll support.  I'd really love it if they would integrate with Gusto.  I was afraid that I would need to set up with Zoho Project and pay another service fee, but it looks -- at least so far from my trial, like this won't be necessary.<br />
520. <br />
521. So my feature request is for integration with Gusto.  In the meantime, I'll probably just handle payroll expenses by manually copying the data from Gusto.<br />
522. <br />
523. <h3>
524. Conclusion</h3>
525. <br />
526. So many, so close, and yet nothing actually hits the mark.   (These aren't all the options I looked at, but they are the main contenders.  Some weren't offered in the US, or were too expensive, or self-hosted.  For now I'm going to try Zoho.  I will try to update this in a few months when I have more experience.<br />
527. <br />
528. <h3>
529. <i>Updates: (As of Nov. 30, 2017) </i></h3>
530. <br />
531. <ol>
532. <li>Zoho has since introduced Zoho Payroll, and they contacted me about it.  It's only available for California at this time, and has some restrictions.  I personally don't want to be an early adopter for my payroll processing service, so I'm going to stick with Gusto for now.   Zoho's representative did tell me that they welcome other payroll processing companies to develop integrations for Zoho Books.   I hope Gusto will take notice.</li>
533. <li>ZipBooks also contacted me.  They apologized for the delays in getting back to me -- apparently their staff left early for Thanksgiving weekend.  They indicated that they have fixed whatever bug caused me to be unable to link my bank account.  Their COO also contacted me, and we had a long phone call, mostly to discuss my thoughts and needs around multi-currency support.  I'm not quite ready to switch to them, but I'd keep a close eye on them.  They do need to work to improve their initial customer service experience, in my opinion.</li>
534. <li>It looks like my own multi-currency needs may be vanishing, as my primary external customer has agreed to be billed in USD and to pay me in USD.  That said, I want to keep the option open for the future, as I may have other international customers in the future.</li>
535. <li>None of the other vendors reached out to me, even though I linked to them on Twitter.  The lack of response itself is "significant" in terms of customer service, IMO. </li>
536. </ol>
537. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/6814406495840619395/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=6814406495840619395' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/6814406495840619395'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/6814406495840619395'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2017/11/small-business-accounting-software-woes.html' title='Small Business Accounting Software Woes'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-8033006439090643695</id><published>2017-11-14T13:06:00.004-08:00</published><updated>2017-11-14T13:14:43.346-08:00</updated><title type='text'>TLS close-notify .... what were they thinking?</title><content type='html'>&lt;h2&gt;
538. Close-Notify Idiocy?</h2>
539. <br />
540. <a href="https://www.ietf.org/rfc/rfc4346.txt">TLS</a> (and presumably SSL) <i><a href="https://tools.ietf.org/html/rfc5246#section-7.2.1">require</a></i> that implementations send a special disconnect message, "close-notify", when closing a connection.  The precise language (from TLS v1.2) reads:<br />
541. <br />
542. <blockquote class="tr_bq" style="break-before: page; font-size: 13.3333px;">
543. The client and the server must share knowledge that the connection is<br />
544. ending in order to avoid a truncation attack.  Either party may<br />
545. initiate the exchange of closing messages. </blockquote>
546. <blockquote class="tr_bq" style="break-before: page; font-size: 13.3333px;">
547. close_notify </blockquote>
548. <blockquote class="tr_bq" style="break-before: page; font-size: 13.3333px;">
549. This message notifies the recipient that the sender will not send<br />
550. any more messages on this connection.  Note that as of TLS 1.1,<br />
551. failure to properly close a connection no longer requires that a<br />
552. session not be resumed.  This is a change from TLS 1.0 to conform<br />
553. with widespread implementation practice. </blockquote>
554. <blockquote class="tr_bq" style="break-before: page; font-size: 13.3333px;">
555. Either party may initiate a close by sending a close_notify alert.<br />
556. Any data received after a closure alert is ignored. </blockquote>
557. <blockquote class="tr_bq" style="break-before: page; font-size: 13.3333px;">
558. Unless some other fatal alert has been transmitted, each party is<br />
559. required to send a close_notify alert before closing the write side<br />
560. of the connection.  The other party MUST respond with a close_notify<br />
561. alert of its own and close down the connection immediately,<br />
562. discarding any pending writes.  It is not required for the initiator<br />
563. of the close to wait for the responding close_notify alert before<br />
564. closing the read side of the connection.</blockquote>
565. <br />
566. This has to be one of the stupider designs I've seen.<br />
567. <br />
568. The stated reason for this is to prevent a "truncation attack", where an attacker terminates the session by sending a clear-text disconnect (TCP FIN) message, presumably just before you log out of some sensitive service, say GMail.<br />
569. <br />
570. The stupid thing here is that this is for WebApps that want to send a logout, and don't want to wait for confirmation that logout had occurred before sending confirmation to the user.  So this logout is unlike every other RPC.  What...?!?<br />
571. <br />
572. <h3>
573. Practical Exploit?</h3>
574. <br />
575. It's not even clear how one would use this attack to compromise a system... an attacker won't be able to hijack the actual TLS session unless they already pwned your encryption.  (In which case, game over, no need for truncation attacks.)  The idea in the truncation attack is that one side (the server?) still thinks the connection is alive, while the other (the browser?) thinks it is closed.  I guess this could be used to cause extra resource leaks on the server... but that's what keep-alives are for, right?<br />
576. <br />
577. <h3>
578. Bugs Everywhere</h3>
579. <br />
580. Of course, close-notify is the source of <i>many</i> bugs (pretty much none of them security critical) in TLS implementations.  Go ahead, Google... I'll wait...  Java, Microsoft, and many others have struggled in implementing this part of the RFC.<br />
581. <br />
582. Even the TLS v1.1 authors recognized that "widespread implementation practice" is simply to ignore this part of the specification and close the TCP channel.<br />
583. <br />
584. So you may be asking yourself, why don't implementations send the close-notify ... after all sending a single message seems pretty straight-forward and simple, right?<br />
585. <br />
586. <h3>
587. Semantic Overreach</h3>
588. <br />
589. Well, the thing is that on many occasions, the application is closing down.  Historically, operating systems would just close() their file descriptors on <span style="font-family: "verdana" , sans-serif;">exit</span>().  Even for long running applications, the quick way to abort a connection is ... <span style="font-family: "verdana" , sans-serif;">close</span>().  With no notification.  Application developers expect that <span style="font-family: "verdana" , sans-serif;">close</span>() is a non-blocking operation on network connections (and most everywhere else)<sup><a href="#footnote1">1</a></sup>.<br />
590. <br />
591. Guess what, you now <i>cannot exit</i> your application without sending this, without breaking the RFC.   That's right, this RFC changes the semantic of <span style="font-family: "verdana" , sans-serif;">exit</span>(2).  Whoa.<br />
592. <br />
593. That's a little presumptive, dontcha think?<br />
594. <br />
595. Requiring implementations to send this message means that now <span style="font-family: "verdana" , sans-serif;">close</span>() grows some kind of new semantic, where the application has to stop and wait for this to be delivered.  Which means TCP has to be flowing and healthy.  The only other RFC compliant behavior is to block and wait for it flow.<br />
596. <br />
597. What happens if the other side is stuck, and doesn't read, leading to a TCP flow control condition?  You <i>can't</i> send the message, because the kernel TCP code won't accept it -- write() would block, and if you're in a non-blocking or event driven model, the event will simply never occur.  Your close() now blocks forever.<br />
598. <br />
599. Defensively, you <b>must</b> insert a timeout somehow -- <i>in violation of the RFC.</i>  Otherwise your TCP session could block forever.  And now you have to contemplate how long to hold the channel open?  You've already decided (for whatever other reason) to abort the session, but you now have to wait a while ... how long is too long?  And meanwhile this open TCP sits around consuming buffer space, an open file descriptor, and perhaps other resources....<br />
600. <br />
601. <h3>
602. A Bit of Sanity</h3>
603. <br />
604. The sensible course of action, treating a connection abort for any reason as an implicit close notification, was simply "not considered" from what I can tell.<br />
605. <br />
606. In my own application protocols, when using TLS, I may violate this RFC <b>with prejudice</b><i><b>. </b>But then I also am not doing stupid things in the protocol like TCP connection reuse.</i>  If you close the connection, all application state with that connection goes away.  Period.  Kind of ... logical, right?<br />
607. <br />
608. Standards bodies be damned.<br />
609. <br />
610. <blockquote>
611. <span style="font-size: x-small;"><a name="footnote1">1.</a> The exception here is historical tape devices, which might actually perform operations like rewinding the tape automatically upon close().  I think this semantic is probably lost in the mists of time for most of us.</span></blockquote>
612. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/8033006439090643695/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=8033006439090643695' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/8033006439090643695'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/8033006439090643695'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2017/11/tls-close-notify-what-were-they-thinking.html' title='TLS close-notify .... what were they thinking?'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-1447562051210370837</id><published>2017-11-08T22:42:00.002-08:00</published><updated>2017-11-08T22:42:50.219-08:00</updated><title type='text'>CMake ExternalProject_add In Libraries</title><content type='html'>First off, I&#39;m a developer of open source application libraries, some of which are fairly popular.&lt;br /&gt;
613. <br />
614. <blockquote class="tr_bq">
615. TLDR: Library developers should <i>not</i> use ExternalProject_Add, but instead rely on FindPackage, demanding that their downstream developers pre-install their dependencies.</blockquote>
616. <br />
617. I recently decided to try to add TLS v1.2 support to one of my messaging libraries, which is written in C and configured via CMake.<br />
618. <br />
619. <div class="separator" style="clear: both; text-align: center;">
620. <a href="https://cmake.org/cmake/img/CMake-JPEG.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="800" data-original-width="800" height="200" src="https://cmake.org/cmake/img/CMake-JPEG.jpg" width="200" /></a></div>
621. <br />
622. <br />
623. The best way for me to do this -- so I thought -- would be to add a dependency in my project using a sub project, bringing in a 3rd party (also open source) library -- <a href="https://tls.mbed.org/">Mbed TLS</a>.<br />
624. <br />
625. Now the Mbed TLS project is also configured by CMake, so you'd think this would be relatively straight-forward to include their work in my own.  You'd be mistaken.<br />
626. <br />
627. CMake includes a capability for configuring external projects, even downloading their source code (or checking out the stuff via git) called <a href="https://cmake.org/cmake/help/v3.0/module/ExternalProject.html">ExternalProjects</a>.<br />
628. <br />
629. This looks super handy -- and it <i>almost</i> is.  (And for folks using CMake to build <i>applications</i> I'm sure this works out well indeed.)<br />
630. <br />
631. Unfortunately, this facility needs a lot of work still -- it only runs at <i>build</i> time, not <i>configuration</i> time.<br />
632. <br />
633. It also isn't immediately obvious that <span style="font-family: Verdana, sans-serif;">ExternalProject_Add()</span> just creates the custom target, without making any dependencies upon that target.  I spent a number of hours trying to understand why my ExternalProject was not getting configured.  Hip hip hurray for CMake's amazing debugging facilities... <i><b>not</b>!  </i>It's sort of like trying to debug some bastard mix of m4, shell, and Python.  Hint, <span style="font-family: Verdana, sans-serif;">Add_Dependencies()</span> is the clue you need, may this knowledge save you hours lack of it cost me.  Otherwise, enjoy the spaghetti.<br />
634. <table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
635. <tr><td style="text-align: center;"><a href="http://www.crazyfreevector.com/uploads/120115/1_155014_1.gif" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="http://www.crazyfreevector.com/uploads/120115/1_155014_1.gif" data-original-height="350" data-original-width="399" height="175" width="200" /></a></td></tr>
636. <tr><td class="tr-caption" style="text-align: center;">Bon Apetit, CMake lovers!</td></tr>
637. </tbody></table>
638. <br />
639. So once you're configuring the dependent library, how are you going to link your own library against the dependent?<br />
640. <br />
641. Well, if you're building an application, you just link (hopefully statically), have the link resolved at compile time, and forget about it forever more.<br />
642. <br />
643. But if you're building a <i>library</i> the problem is harder.  You can't include the dependent library directly in your own.  There's no portable way to "merge" archive libraries or even dynamic libraries.<br />
644. <br />
645. Basically, your consumers are going to be stuck having to link against the dependent libraries as well as your own (and in the right order too!)  You want to make this easier for folks, but you just <i>can't. </i><br />
646. <i>(My kingdom for a C equivalent to the Golang solution to this problem.  No wonder Pike et. al. got fed up with C and invented Go!)</i><br />
647. <i><br /></i>
648. <table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
649. <tr><td style="text-align: center;"><a href="http://rs717.pbsrc.com/albums/ww173/prestonjjrtr/Funny/GopherYayDance.gif~c200" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="http://rs717.pbsrc.com/albums/ww173/prestonjjrtr/Funny/GopherYayDance.gif~c200" data-original-height="200" data-original-width="200" height="200" width="200" /></a></td></tr>
650. <tr><td class="tr-caption" style="text-align: center;">And Gophers everywhere rejoiced!</td></tr>
651. </tbody></table>
652. <br />Making matters worse, the actual library (or more, as in the aforementioned TLS software there are actually <i>3</i> separate libraries -- libmbedcrypto, libmbedx509, and libmbedtls) is located somewhere deeply nested in the build directory.   Your poor consumers are never gonna be able to figure it out.<br />
653. <br />
654. There are two solutions:<br />
655. <br />
656. a) Install the dependency as well as your own library (and tell users where it lives, perhaps via pkgconfig or somesuch).<br />
657. <br />
658. b) Just forget about this and make users pre-install the dependency explicitly themselves, and pass the location to your configuration tool (CMake, autotools, etc.) explicitly.<br />
659. <br />
660. Of these two, "a" is easier for end users -- as long as the application software doesn't also want to use functions in that library (perhaps linking against a *different* copy of the library).  If this happens, the problem can become kind of intractable to solve.<br />
661. <br />
662. So, we basically punt, and make the user deal with this.  Which tests days for many systems is handled by packaging systems like debian, pkg-add, and brew.<br />
663. <br />
664. After having worked in Go for so long (and admittedly in kernel software, which has <i>none</i> of these silly userland problems), the current state of affairs here in C is rather disappointing.<br />
665. <br />
666. Does anyone out there have any other better ideas to handle this (I mean besides &quot;develop in &lt;i&gt;Y&lt;/i&gt;&quot;, where &lt;i&gt;Y&lt;/i&gt; is some language besides C)?</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/1447562051210370837/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=1447562051210370837' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/1447562051210370837'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/1447562051210370837'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2017/11/cmake-externalprojectadd-in-libraries.html' title='CMake ExternalProject_add In Libraries'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-4819076148015241232</id><published>2017-11-08T22:07:00.001-08:00</published><updated>2017-11-08T22:07:45.708-08:00</updated><title type='text'>Licensing... again....</title><content type='html'>Let me start by saying this... I hate the GPL.&amp;nbsp; Oh yeah, and a heads up, I am just a software engineer, and not a lawyer.&amp;nbsp; Having said that....&lt;br /&gt;
667. <br />
668. I've released software under the GPL, but I never will again.  Don't get me wrong, I love open source, but GPL's license terms are unaccountably toxic, creating an island that I am pretty sure that original GPL authors never intended.<br />
669. <br />
670. <div class="separator" style="clear: both; text-align: center;">
671. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOms5D4OleF27z5f2pP4_BG3M9vSOFYKqWkGZa_hHNlTHJTO4iOGfqm6LiNlA0SRYZlNgKb0koeTsLEpWsGlBLF7pX3LRhp_zeu7fyAAxQvYNFgYrrY8n7QZ28hWZ1MZEpWDyGy9ABh4w/s1600/antignu.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="525" data-original-width="507" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOms5D4OleF27z5f2pP4_BG3M9vSOFYKqWkGZa_hHNlTHJTO4iOGfqm6LiNlA0SRYZlNgKb0koeTsLEpWsGlBLF7pX3LRhp_zeu7fyAAxQvYNFgYrrY8n7QZ28hWZ1MZEpWDyGy9ABh4w/s200/antignu.png" width="193" /></a></div>
672. <br />
673. <h4>
674. My Problem....</h4>
675. <br />
676. So I started by wanting to contemplate a licensing change for a new library I'm working on, to move from the very loose and liberal <a href="https://directory.fsf.org/wiki/License:X11">MIT</a> license, to something with a few characteristics I like -- namely patent protection and a "builtin" contributor agreement.   I'm speaking of course of the well-respected and well-regarded <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License 2.0.</a><br />
677. <br />
678. The problem is, I ran into a complete and utter roadblock.<br />
679. <br />
680. I want my software to be maximally usable by as many folks as possible.<br />
681. <br />
682. There is a large installed base of software released under the <a href="https://www.gnu.org/licenses/gpl-2.0.html">GPLv2</a>.  (Often without the automatic upgrade clause.)<br />
683. <br />
684. Now I'm not a big fan of "viral licenses" in general, but I get that folks want to have a copy-left that prevents folks from including their work in closed source projects.  I get it, and it's not an entirely unreasonable position to hold, even if I think it limits adoption of such licensed software.<br />
685. <br />
686. My problem is, that the GPLv2's terms are incredibly strict, prohibiting any other license terms being applied by any other source in the project.  This means that you can't mix GPLv2 with pretty much <i>anything</i> else, except the very most permissive licenses.  The patent grant & protection clauses breaks GPLv2.  (In another older circumstance, the CDDL had similar issues which blocks ZFS from being distributed with the Linux kernel proper.  The CDDL also had a fairly benign choice-of-venue clause for legal action, which was also deemed incompatible to the GPLv2.)<br />
687. <br />
688. So at the end of the day, GPLv2 freezes innovation and has limited my <i>own actions</i> because I would like to enable people who have GPLv2 libraries to use my libraries.  We even have an ideological agreement -- the <a href="https://www.gnu.org/licenses/license-list.html#apache2">FSF actually <i>recommends</i> the Apache License 2.0!</a>  And yet I can't use it; I'm stuck with a very much inferior MIT license in order to let GPLv2 folks play in the pool.<br />
689. <br />
690. Wait, you say, what about the <a href="https://www.gnu.org/licenses/gpl-3.0.en.html">GPLv3</a>?  It fixed these incompatibilities, right?   Well, yeah, but then it went and added <i>other</i> constraints on use which are even more chilling than the GPLv2.  (The <a href="http://www.linfo.org/tivoization.html">anti-Tivoization</a> clause, which is one of the more bizarre things I've seen in any software license, applying only to equipment intended primarily "consumer premises".  What??)<br />
691. <br />
692. The GPL is the FOSS movements worst enemy, in my opinion.  Sure, Linux is everywhere, but I believe that this is in <i>spite</i> of the GPLv2 license, rather than as a natural by product.  The same result could have been achieved under a liberal, or a file-based copyleft.<br />
693. <br />
694. <h4>
695. GPL in Support of Proprietary Ecosystems</h4>
696. <br />
697. In another turn of events, the GPL is now being used by <i>commercial</i> entities in a bait-and-switch.  In this scheme, they hook the developer on their work under the GPL.  But when the developer wants to add some kind of commercial capability and retain the source confidentially, the developer cannot do that -- <i>unless the developer pays the original author a fee for a special commercial license.</i>    For a typical example, have a look at <a href="https://www.wolfssl.com/license/">the WolfSSL license page</a>.<br />
698. <br />
699. Now all that is fine and dandy legal as you please.  But, in this case, the GPL isn't being used to promote open source at all.  Instead, it has become an enabler for monetization of closed source, and frankly leads to a richer <i>proprietary</i> software ecosystem.  I don't think this was what the original GPL authors had intended.<br />
700. <br />
701. Furthermore, because the author of this proprietary software needs to be able to relicense the code under commercial terms, they are very very unlikely to accept contributions from third parties (e.g. external developers) -- unless those contributors are willing to perform a copyright assignment or sign a contributor agreement giving the commercial entity very broad relicensing rights.<br />
702. <br />
703. So instead of becoming an enabler for open collaboration, the GPL just becomes another tool in the pockets of commercial interests.<br />
704. <br />
705. The GPL Needs to Die<br />
706. <br />
707. If you love open source, and you want to enhance innovation, please, &lt;i&gt;please&lt;/i&gt;&amp;nbsp;don&#39;t license your stuff under GPL unless you have no other choice.&amp;nbsp; If you can relicense your work under other terms, please do so!&amp;nbsp; Look for a non-viral license with the patent protections needed for both your and your downstreams.&amp;nbsp; I recommend either the Mozilla Public License (if you need a copyleft on your own code), or the Apache License (which is liberal but offers better protections over BSD or MIT or similar alternatives.)</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/4819076148015241232/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=4819076148015241232' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4819076148015241232'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4819076148015241232'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2017/11/licensing-again.html' title='Licensing... again....'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOms5D4OleF27z5f2pP4_BG3M9vSOFYKqWkGZa_hHNlTHJTO4iOGfqm6LiNlA0SRYZlNgKb0koeTsLEpWsGlBLF7pX3LRhp_zeu7fyAAxQvYNFgYrrY8n7QZ28hWZ1MZEpWDyGy9ABh4w/s72-c/antignu.png" height="72" width="72"/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-4932888243302548895</id><published>2016-10-24T21:53:00.000-07:00</published><updated>2016-10-24T21:53:03.943-07:00</updated><title type='text'>MacOS X Mystery (Challenge)</title><content type='html'>(Maybe my MacOS X expert friends will know the answer.)&lt;br /&gt;
708. <br />
709. This is a mystery that I cannot seem to figure out.  I think its a bug in the operating system, but I cannot seem to figure out the solution, or even explain the behavior to my satisfaction.<br />
710. <br />
711. Occasionally, a shell window (iTerm2) will appear to "forget" my identity. <br />
712. <br />
713. For example:<br />
714. <br />
715. <span style="font-family: Courier New, Courier, monospace;">% whoami</span><br />
716. <span style="font-family: Courier New, Courier, monospace;">501</span><br />
717. <br />
718. That's half right... The same command in other window is more correct:<br />
719. <br />
720. <span style="font-family: Courier New, Courier, monospace;">% whoami</span><br />
721. <span style="font-family: Courier New, Courier, monospace;">garrett</span><br />
722. <br />
723. Further, id -a reports differently:<br />
724. <br />
725. The broken window:<br />
726. <br />
727. <span style="font-family: Courier New, Courier, monospace;">% id -a</span><br />
728. <span style="font-family: Courier New, Courier, monospace;">uid=501 gid=20(staff) groups=20(staff),501,12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),33(_appstore),100(_lpoperator),204(_developer),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh)</span><br />
729. <br />
730. The working one:<br />
731. <br />
732. <span style="font-family: Courier New, Courier, monospace;">% id -a</span><br />
733. <span style="font-family: Courier New, Courier, monospace;">uid=501(garrett) gid=20(staff) groups=20(staff),501(access_bpf),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),33(_appstore),100(_lpoperator),204(_developer),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh)</span><br />
734. <br />
735. It appears that the shell (and this broken behavior seems to be inherited by child shells, by the way), somehow loses the ability to map numeric Unix ids to login names.<br />
736. <br />
737. So I tried another command:<br />
738. <br />
739. <span style="font-family: Courier New, Courier, monospace;">% dscl . -read /Users/garrett</span><br />
740. <span style="font-family: Courier New, Courier, monospace;">Operation failed with error: eServerError</span><br />
741. <span style="font-family: Courier New, Courier, monospace;"><br /></span>
742. The same works properly in my other window (I'm not posting the entire output, since its really long).<br />
743. <br />
744. I am wondering what could possibly be different.  The behavior doesn't seem to depend on environment variables (I've tried stripping those out).<br />
745. <br />
746. I'm thinking that there is something in the process table (in the MacOS X equivalent of the uarea?) that gives me access to directory services -- and that this is somehow clobbered.  As indicated, whatever the thing is, it appears to be inherited across fork(2).<br />
747. <br />
748. I thought maybe I could figure this out with DTrace or dtruss... but Apple have crippled DTrace on the platform and this is one of those binaries that I am unable to introspect.  Arrgh!<br />
749. <br />
750. <span style="font-family: Courier New, Courier, monospace;">sudo dtruss dscl . -read /Users/garrett</span><br />
751. <span style="font-family: Courier New, Courier, monospace;">Password:</span><br />
752. <span style="font-family: Courier New, Courier, monospace;">dtrace: system integrity protection is on, some features will not be available</span><br />
753. <span style="font-family: Courier New, Courier, monospace;"><br /></span>
754. <span style="font-family: Courier New, Courier, monospace;">dtrace: failed to execute dscl: dtrace cannot control executables signed with restricted entitlements</span><br />
755. <span style="font-family: Courier New, Courier, monospace;"><br /></span>
756. Btw, I'm running the latest MacOS X:<br />
757. <br />
758. <span style="font-family: Courier New, Courier, monospace;">% uname -a</span><br />
759. <span style="font-family: Courier New, Courier, monospace;">Darwin Triton.local 16.0.0 Darwin Kernel Version 16.0.0: Mon Aug 29 17:56:20 PDT 2016; root:xnu-3789.1.32~3/RELEASE_X86_64 x86_64</span><br />
760. <br />
761. So, for my MacOS X expert friends -- anyone know how directory services <i>really</i> works?  (As in how it works under the hood?)  I don't think we're in UNIX land anymore, Toto!<br />
762. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/4932888243302548895/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=4932888243302548895' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4932888243302548895'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4932888243302548895'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2016/10/macos-x-mystery-challenge.html' title='MacOS X Mystery (Challenge)'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-4956875839204442298</id><published>2016-10-24T11:44:00.002-07:00</published><updated>2016-10-24T11:44:58.610-07:00</updated><title type='text'>Security Advice to IoT Firmware Engineers</title><content type='html'>Last Friday (October 16, 2016), a &lt;a href=&quot;https://www.wired.com/2016/10/internet-outage-ddos-dns-dyn/&quot;&gt;major DDoS&lt;/a&gt; attack brought down a number of sites across the Internet. &amp;nbsp;My own &lt;a href=&quot;http://www.saucelabs.com/&quot;&gt;employer&lt;/a&gt; was amongst those affected by the wide spread DNS outage.&lt;br /&gt;
763. <br />
764. It turns out that the sheer scale (millions of unique botnet members) was made possible by the IoT, and rather shoddy engineering practices.<br />
765. <br />
766. Its time for device manufacturers and firmware engineers to "grow up", and learn how to properly engineer these things for the hostile Internet, so that they don't have to subsequently issue recalls when their customers' devices are weaponized by attackers without their owners knowledge.<br />
767. <br />
768. This blog is meant to offer some advice to firmware engineers and manufacturers in the hope that it may help them prevent their devices from being used in these kinds of attacks in the future.<br />
769. <h4>
770. <br /></h4>
771. <h4>
772. Passwords</h4>
773. <div>
774. <br /></div>
775. <div>
776. Passwords are the root of most of the problems, and so much of the advice here is about improving the way these are handled.</div>
777. <h4>
778. <br /></h4>
779. <h4>
780. No Default Passwords</h4>
781. <br />
782. The idea of using a simple default password and user name, like "admin/admin", is a practice from the 90's, and is intended to facilitate service personnel, and eliminate management considerations from dealing with many different passwords.  Unfortunately, this is probably the single biggest problem -- bad usernames and passwords.  Its far worse in an IoT world, where there are many thousands, or even millions, of devices that have the same user name and password.<br />
783. <br />
784. The proper solution is to allocate a unique password to each and every device.  Much like we already do manage unique MAC addresses, we need every device to have a unique password.  (Critically, the password must <i>not</i> be derived from the MAC address though.)<br />
785. <br />
786. My advice is to simply have a small amount of ROM that is factory burned with either a unique password, or a numeric key that can be used to create one.  (If you have enough memory to store a dictionary in generic firmware -- say 32k words, you can get very nice human manageable default passwords by storing just four 16-bit numbers, each representing an index into the dictionary (so only 15 bits of unique data, but thats 60 bits of total entropy, which is plenty to ensure that every device has its own password -- and only requires storing a 64-bit random number in ROM.)<br />
787. <br />
788. Then you have nice human parseable passwords like "bigger-stampede-plasma-pandering".  These can be printed on the same sticker that MAC passwords are typically given.  (You could also accept a hexadecimal representation of the underlying 64-bit value, or just use that instead of human readable passwords if you are unable to accommodate an English dictionary.  Devices localized for use in other countries could use locale-appropriate dictionaries as well.)<br />
789. <h4>
790. <br /></h4>
791. <h4>
792. Mandatory Authorization Delay</h4>
793. <br />
794. Second, IoT devices should inject a minimum delay after password authentication attempts (regardless of whether successful or otherwise).  Just a few seconds is enough to substantially slow down dictionary attacks against poorly chosen end-user passwords.  (2 seconds means that only 1800 unique attempts can be performed per hour under automation - 5 seconds reduces that to 720.  It will be difficult to iterate a million passwords against a device that does this.)<br />
795. <h4>
796. <br /></h4>
797. <h4>
798. Strong Password Enforcement</h4>
799. <br />
800. User chosen passwords should not be a single dictionary word; indeed, the default should be to use a randomly generated password using the same dictionary approach above (generate a 64-bit random number, break into chunks, and index into a stock dictionary).  It may be necessary to provide an end-user override, but it should be somewhat difficult to get at by default, and when activate should display large warnings about the compromise to security that user-chosen passwords typically represent.<br />
801. <h4>
802. <br /></h4>
803. <h4>
804. Networks</h4>
805. <div>
806. <br /></div>
807. <div>
808. Dealing with the network, and securing the use of the network, is the other part of the problem that IoT vendors need to get right.</div>
809. <h4>
810. <br /></h4>
811. <h4>
812. Local Network Authentication Only</h4>
813. <br />
814. IoT devices generally know the network they are on; if the device has a separate management port or LAN-only port (like a WiFi Router), it should only by default allow administrator access from that port.<br />
815. <br />
816. Devices with only a single port, or that exist on a WiFi network, should prevent administrator access from "routed" networks, by default.   That is, devices should not allow login attempts from a remote IP address that is not on a local subnet, by default.  While this won't stop many attacks (especially those on public WiFis), it makes attacking them from a global botnet, or managing them as part of a global botnet, that much harder.   (Again, there has to be a provision to disable this limitation, but it should present a warning.)<br />
817. <h4>
818. <br /></h4>
819. <h4>
820. Encrypted Access Only</h4>
821. <br />
822. Use of unsecured channels (HTTP or telnet) is unacceptable in this day and age.  TLS and/or SSH are the preferred ways to do this, and will let your customers deploy these devices somewhat more securely.<br />
823. <br />
824. <h4>
825. Secure All Other Ports</h4>
826. <br />
827. Devices should disable any network services that are not specifically part of the service they offer, or intrinsic to their management.   System administrators have known to do this on systems for decades now, but it seems some firmwares still have stock services enabled that can be used as attack vectors.<br />
828. <h4>
829. <br /></h4>
830. <h4>
831. Don't Advertise Yourself</h4>
832. <br />
833. This one is probably the hardest.  <a href="https://threatpost.com/multicast-dns-vulnerability-could-lead-to-ddos-amplification-attacks/111936/">mDNS</a> and device discovery over "standard" networks is one of the ways that attackers find devices to target.  Its far far better to have this disabled by default -- if discovery is needed during device configuration, then it can be enabled briefly, when the device is being configured.  Having a "pairing" button to give end-users the ability to enable this briefly is useful -- but mDNS should be used only with caution.<br />
834. <h4>
835. <br /></h4>
836. <h4>
837. Secure Your Channel Home</h4>
838. <br />
839. Devices often want to call-home for reporting, or web-centric command & control.  (E.g. remote management of your thermostat.)  This is one of the major attack vectors.   (If you can avoid calling home altogether, this is even better!)<br />
840. <br />
841. Users must be able to disable this function (it should be disabled by default in fact).  Furthermore, the channels must be properly secured entirely through your network, with provision for dealing with a compromise (e.g. leaked private keys at the server side).  Get a security expert to review your protocols, and your internal security practices.<br />
842. <h4>
843. <br /></h4>
844. <h4>
845. Mesh Securely</h4>
846. <br />
847. Building local mesh networks of devices, e.g. to create a local cloud, means having strong pairing technology.  The strongest forms of this require administrator action to approve -- just like pairing a bluetooth keyboard or other peripheral.<br />
848. <br />
849. If you want to automate secure mesh provisioning, you have to have secure networking in place -- technologies like VPN or <a href="https://www.zerotier.com/">ZeroTier</a> can help build networking layers that are secure by default.<br />
850. <h4>
851. <br /></h4>
852. <h4>
853. Don't Invent Your Own Protocols</h4>
854. <br />
855. The roadside is littered with the corpses of protocols and products that attempted to invent their own protocols or use cryptography in non-standard ways.  The best example of this is <a href="https://en.wikipedia.org/wiki/Wired_Equivalent_Privacy">WEP</a>, which took a relatively secure crypto layer (RC4 was not broken at the time), but deployed it naively and brokenly.  RC4 got a very bad rap for this, but it was actually WEP that was broken.  (Since then, RC4 itself has been shown to have some weaknesses, but this is relatively new compared to the brokenness that was WEP.) <br />
856. <h4>
857. <br /></h4>
858. <h4>
859. General Wisdoms</h4>
860. <div>
861. <br /></div>
862. <div>
863. Next we have some advice that most people should already be aware of, but yet bears repeating.</div>
864. <h3>
865. <br /></h3>
866. <h4>
867. Don't Rely on Obscurity</h4>
868. <br />
869. Its an old adage that "security by obscurity is no security at all".  Yet we often see naive engineers trying to harden systems by making them more obscure.  This really doesn't help anything long term, and can actually hinder security efforts by giving a false sense of security or creating barriers to security analysis.<br />
870. <h4>
871. <br /></h4>
872. <h4>
873. Audit</h4>
874. <br />
875. Get an independent security expert to audit your work.  Special focus should be paid to the items pointed out above.  This should include a review of the product, as well as your internal practices around engineering, including secure coding, use of mitigation technologies, and business practices for dealing with keying material, code signing, and other sensitive data.<br />
876. <div>
877. <br /></div>
878. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/4956875839204442298/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=4956875839204442298' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4956875839204442298'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4956875839204442298'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2016/10/security-advice-to-iot-firmware.html' title='Security Advice to IoT Firmware Engineers'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-4945004197183723437</id><published>2016-05-14T09:24:00.000-07:00</published><updated>2016-05-14T09:24:35.297-07:00</updated><title type='text'>Microsoft Hates My Name (Not Me, Just My Name)</title><content type='html'>In order to debug &lt;a href=&quot;http://www.nanomsg.org/&quot;&gt;nanomsg&lt;/a&gt; problems on Windows, I recently installed a copy of Windows 8.1 in a VMWare guest VM, along with Visual Studio 14 and CMake 3.5.2. &amp;nbsp;(Yes, I&#39;ve entered a special plane of Hell, reserved for just for people who try to maintain cross-platform open source software. &amp;nbsp;I think this one might be the tenth plane, that Dante skipped because it was just too damned horrible.)&lt;div&gt;
879. <br /></div>
880. <div>
881. Every time I tried to build, I got bizarre errors from the CMake / build process ... like this:</div>
882. <div>
883. <br /></div>
884. <div>
885.  
886.  
887.  
888.  
889.  
890.  
891.  
892. <blockquote class="tr_bq">
893. <span style="font-family: Courier New, Courier, monospace;">Cannot evaluate the item metadata "%(FullPath)</span></blockquote>
894. <div class="p1">
895. <br /></div>
896. <div class="p1">
897. Turns out that when I created my account, using the "easy" installation in VMWare, it created my Windows account using my full name.  "<b>Garrett D'Amore</b>".  Turns out that the software is buggy, and can't cope with the apostrophe in my full name, when it appears in a filesystem path. </div>
898. <div class="p1">
899. <br /></div>
900. <div class="p1">
901. Moving the project directory to C:\Projects\nanomsg solved the problem.</div>
902. <div class="p1">
903. <br /></div>
904. <div class="p1">
905. Really Microsoft?  This is 2016.  I expected programs to struggle and for me to find bugs in programs (often root exploits  -- all hackers should try using punctuation in their login and personal names) with the apostrophe in my name back in the 1990s.  Not in this decade.</div>
906. <div class="p1">
907. <br /></div>
908. <div class="p1">
909. Not only that, but the error message was so incredibly cryptic that it took a Google search to figure out that it was a problem with the path.  (Other people encountered this problem with paths > 260 characters.  I knew that wasn't my problem, but I hypothesized, and proved, that it was my name.)  I have no idea how to file a bug on Visual Studio to Microsoft.  I'm not a paying user of it, so maybe I shouldn't complain, and I really have no recourse.  Still, they need to fix this.</div>
910. <div class="p1">
911. <br /></div>
912. <div class="p1">
913. Normally, I'd never intentionally create a path with an apostrophe in it, but in this case I was being lazy and just accepted some defaults.  I staunchly refuse to change my <i>name</i> because some software is too stupid to cope with it -- this is a pet peeve for me. </div>
914. <div class="p1">
915. <br /></div>
916. <div class="p1">
917. We're in the new millennium, and have been for a decade and half.  Large numbers of folks with heritage from countries like Italy, France, and Ireland have this character in their surname.  (And more recently -- since like the 1960s! -- the African-American community has been using this character in their <b>first</b> names too!)  If your software can't accommodate this common character in names, then it's broken, and you need to fix it.  There are literally millions of us that are angered by this sort of brokenness every day; do us all a favor and make your software just a little less rage inducing by letting us use the names we were born with please.</div>
918. <div class="p1">
919. <br /></div>
920. <div class="p1">
921. <br /></div>
922. <div class="p1">
923. <br /></div>
924. </div>
925. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/4945004197183723437/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=4945004197183723437' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4945004197183723437'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4945004197183723437'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2016/05/microsoft-hates-my-name-not-me-just-my.html' title='Microsoft Hates My Name (Not Me, Just My Name)'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-8365365563163650994</id><published>2016-02-23T12:03:00.000-08:00</published><updated>2016-02-23T12:03:13.379-08:00</updated><title type='text'>Leaving github</title><content type='html'>&lt;div class=&quot;p1&quot;&gt;
926. <span class="s1">(Brief reminder that this represents my own personal opinion, not necessarily that of any employer or larger open source project.)</span></div>
927. <div class="p1">
928. <span class="s1"><br /></span></div>
929. <div class="p1">
930. <span class="s1">I am planning to move my personal git repositories (including <a href="http://github.com/go-mangos/mangos">mangos</a>, <a href="http://github.com/gdamore/tcell">tcell</a>, <a href="http://github.com/gdamore/govisor">govisor</a>, <a href="http://github.com/gdamore/less-fork">less-fork</a>, etc.)  from <a href="http://github.com/">GitHub</a> to <a href="http://gitlab.com/">GitLab.com</a>. </span></div>
931. <div class="p2">
932. <span class="s1"></span><br /></div>
933. <div class="p1">
934. <span class="s1">The reasons for this are fairly simple.  They have nothing to do whatsoever with technology.  I love the GitHub platform, and have been a happy user of it for years now. I would dearly love it if I could proceed with GitHub.  Fortunately GitLab seems to have <a href="https://about.gitlab.com/comparison/">feature parity</a> with GitHub (and a growing user and project base), so I'm not trapped.</span></div>
935. <div class="p2">
936. <br /></div>
937. <div class="p2">
938. The reason for leaving GitHub is because of the <a href="http://beforeitsnews.com/opinion-conservative/2016/02/githubs-anti-white-agenda-3102938.html">hostility</a> of it's leadership towards certain classes of people makes me feel that I cannot in good conscience continue to support them. In particular, their HR department is <a href="http://www.techinsider.io/github-the-full-inside-story-2016-2">engaging</a> in what is nothing less than race <a href="http://static4.businessinsider.com/image/56b3d2f12e526555008b505e-791-385/danilo%20campos-tweet.png">warfare</a> against white people.  (Especially men, but even white women are being discriminated against.) By the way, I'd take the same position if the hostility were instead towards any other racial or gender group other than my own.</div>
939. <div class="p2">
940. <br /></div>
941. <div class="p1">
942. <span class="s1">I'm not alone in asking GitHub to fix this; yet they've remained silent on the matter, leading me to believe that the problematic policies have support within the highest levels of the company.  (Github itself is <a href="http://www.businessinsider.com/github-identity-crisis-2016-2">in trouble</a>, and I have doubts about its <a href="http://thenextweb.com/dd/2016/02/04/reports-indicate-github-is-having-major-issues-regarding-the-companys-future/#gref">future</a>, as both <a href="http://thenextweb.com/dd/2016/01/19/frustrated-developers-from-the-webs-biggest-projects-say-github-isnt-listening/">developers</a> and employees are leaving in droves.)</span></div>
943. <div class="p1">
944. <span class="s1"><br /></span></div>
945. <div class="p1">
946. <span class="s1">Post <a href="https://en.wikipedia.org/wiki/Tom_Preston-Werner">Tom Preston-Werner</a>, GitHub's leadership apparently sees the company as a platform for prosecuting the Social Justice War, and it even has a <a href="http://cs.sfsu.edu/jobs/attachments/15/Social_Impact_Intern.pdf">Social Impact Team</a> just to that effect. </span><span class="s1">In GitHub's own words:</span></div>
947. <blockquote class="tr_bq">
948. <span class="s1">"T</span>he Social Impact team will be focused
949. on these three areas: - Diversity & Inclusion - both internally and within the Open Source Community - Community
950. Engagement - we have a net positive impact in local and online communities via partnerships - Leveraging GitHub
951. for Positive Impact - supporting people from varied communities to use GitHub.com in innovative ways"</blockquote>
952. <div class="p2">
953. <span class="s1"></span>It's no accident that they list "Diversity & Inclusion" as the first item here either.  Apparently this has been <a href="https://github.com/dear-github/dear-github">more of a priority for GitHub than improving their platform</a> or addressing long standing customer issues.</div>
954. <div class="p2">
955. <br /></div>
956. <div class="p1">
957. <span class="s1">Those of you who have followed me know that I’m strongly in favor of <i><b>inclusion</b></i>, and making an environment friendly for <i><b>all</b></i> people, regardless of race or gender or religion (provided your religion respects my basic rights -- religious fundamentalist nut-jobs need not apply).</span></div>
958. <div class="p1">
959. <br /></div>
960. <div class="p2">
961. Lack of diversity cannot be fixed through exclusion.  Attempts to do so are inherently misguided.  Furthermore, as a company engages in <i>any exclusive hiring practices</i> they are inherently limiting their own access to talent.  Racist or sexist (or ageist) approaches are self-destructive, and companies that engage in such behavior deserve to fail.</div>
962. <div class="p2">
963. <br /></div>
964. <div class="p2">
965. The way to fix an un-level playing field is to level the playing field -- not to swing it back in the other direction.  You can't fix social injustice with more injustice; we should guarantee equal opportunity not equal results.</div>
966. <div class="p2">
967. <br /></div>
968. <div class="p2">
969. There are plenty of people of diverse ethnic backgrounds who have overcome significant social and economic barriers to achieve success.  And many who have not.  News flash -- you will find white men and women in both lists, as well as blacks, latinos, women, gays, and people of "other gender identification".  Any hiring approach or policy (written or otherwise) that only looks at the color of a person's skin or gender is unfair, and probably illegal outside of a very limited few and specific instances (e.g. casting for movie roles).</div>
970. <div class="p2">
971. <br /></div>
972. <div class="p2">
973.  Note that this does not mean that I do not support efforts to reach out to encourage people from other groups to engage more in technology (or any other field).  As I said, I encourage efforts to include <i>everyone</i> -- the larger talent pool that we can engage with, the more successful we are likely to be.  And we should do everything we can as a society and as an industry to make sure that the talent pool is as big as we can make it.</div>
974. <div class="p2">
975. <br /></div>
976. <div class="p2">
977. We should neither exclude any future <a href="https://en.wikipedia.org/wiki/Marie_Curie">Marie Curie</a> or <a href="http://www.biography.com/people/daniel-hale-williams-9532269">Daniel Hale Williams</a> from achieving the highest levels of success, nor should we exclude a future <a href="https://en.wikipedia.org/wiki/Isaac_Newton">Isaac Newton</a> just because of his race or gender.  The best way to avoid that, is to be inclusive of everyone, and make sure that everyone has the best opportunities to achieve success possible.</div>
978. <div class="p2">
979. <br /></div>
980. <div class="p1">
981. <span class="s1">Sadly I will probably be labeled racist or sexist, or some other -ist, because I'm not supportive of the divisive agendas supported by people like <a href="http://www.usatoday.com/story/tech/columnist/2015/02/12/women-of-color-diversity-tech-silicon-valley-nicole-sanchez/23298945/">Nicole Sanchez</a> and Danilo Libre, and because I am a heterosexual white middle class male (hence automatically an entitled enemy in their eyes.)  It seems that they would rather have me as an enemy rather than a friendly supporter -- at least that is what their actions demonstrate.  It's certainly easier to apply an -ist label than to engage in rationale dialogue.</span></div>
982. <div class="p1">
983. <span class="s1"><br /></span></div>
984. <div class="p1">
985. <span class="s1">I am however deeply supportive of efforts to reach out to underrepresented groups in early stages.  Show more girls, blacks, and latinos filling the role of technophiles in popular culture (movies and shows) that market towards children.  Spend money (wisely!) to improve education in poorer school districts.  Teach kids that they truly can be successful regardless of color or gender, and make sure that they have the tools (including access to technology) to <i>achieve success based on merit, not because of their grouping.</i>  These efforts have to be made at the primary and secondary school levels, where inspiration can have the biggest effects.  (By the way, these lessons apply equally well to white boys; teaching children to respect one another as individuals rather than as labels is a good thing, in all directions.)</span></div>
986. <div class="p1">
987. <span class="s1"><br /></span></div>
988. <div class="p1">
989. <span class="s1">By the time someone in is choosing a college or sitting in front of a recruiter, it's far too late (and far too expensive).  The only tools that can be applied at later stages are only punitive in nature, and therefore the only reasonable thing to do at this late stage is to punish unjust behaviors (i.e. zero tolerance towards bigotry, harassment, and so forth.)</span></div>
990. <div class="p1">
991. <br /></div>
992. <div class="p2">
993. I'll have more detail as to the moves of the specific repos over the coming days.</div>
994. <div class="p2">
995. <br /></div>
996. <div class="p2">
997. PS:  GitLab does <a href="https://about.gitlab.com/2016/02/02/gitlab-diversity-sponsorship/">support</a> diversity as well, which is a good thing, but they do it without engaging in the social justice war, or exclusive policies.</div>
998. <div>
999. <br /></div>
1000. <div class="p2">
1001. <br /></div>
1002. </content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/8365365563163650994/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=8365365563163650994' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/8365365563163650994'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/8365365563163650994'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2016/02/leaving-github.html' title='Leaving github'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-3668235664676989390</id><published>2016-01-06T10:43:00.000-08:00</published><updated>2017-11-08T22:47:22.613-08:00</updated><title type='text'>Stepping Down</title><content type='html'>&lt;blockquote class=&quot;tr_bq&quot;&gt;
1003. <i>Updated Nov 9, 2017: When I originally posted this, nearly two years ago, things were different.  As folks may know, I returned back to leadership of nanomsg, and have since released several significant updates including version 1.0.0 and follow ups.  I'm also in the process of a complete architectural redesign and rewrite (<a href="https://github.com/nanomsg/nng">nng</a>), which is fast nearing completion.  This post is left here for posterity, but if you've wandered here via search-engine, be certain that the nanomsg community is alive and well.</i></blockquote>
1004. <br />
1005. (Quick reminder that this blog represents my own opinion, and not necessarily that of any open source project or employer.)<br />
1006. <br />
1007. For nearly a year, I've been primary maintainer of <a href="http://www.nanomsg.org/">nanomsg</a>, a library of common lightweight messaging patterns written in C.<br />
1008. <br />
1009. I was <a href="https://www.freelists.org/post/nanomsg/New-fork-was-Re-Moving-forward,1">given</a> this mantle when I <a href="https://www.freelists.org/post/nanomsg/Moving-forward">asked</a> for the nanomsg community to take some action to get forward progress on some changes I had to fix some core bugs, one of which was a protocol bug.  (I am also the creator of <a href="https://github.com/go-mangos">mangos</a>, a wire-compatible library supporting the same patterns written in Go, which is why I came to care about fixing nanomsg.)<br />
1010. <br />
1011. Today, I am stepping down as maintainer.<br />
1012. <br />
1013. There are several reasons for this, but the most relevant right now is my frustration with this community, and its response to what I believed to be a benign <a href="https://www.freelists.org/post/nanomsg/Adopting-a-CoC">proposal</a>, that to adopt a <a href="http://contributor-covenant.org/version/1/3/0/code_of_conduct.md">Code of Conduct</a>, in an attempt to make the project more inviting to a broader audience.<br />
1014. <br />
1015. I was unprepared for the backlash.<br />
1016. <br />
1017. And frankly, I haven't got enough love of the project to want to continue to lead it, when its clearly unwilling to codify what are frankly some sound and reasonable communication practices.<br />
1018. <br />
1019. As maintainer, I could have just enforced my will upon the project, but since the project existed before I came to it, that doesn't feel right.  So instead, I'm just stepping down.<br />
1020. <br />
1021. I'm not sure who will succeed me.  I can nominate a party, but at this point there are several other parties with git commit privileges to the project; I think they should nominate one.  Martin (the founder) still has administrative privileges as well.<br />
1022. <br />
1023. <i>To be clear, I think both sides of the Code of Conduct are <b>wrong</b> -- a bunch of whinny kids really. </i><br />
1024. <i><br /></i>
1025. On the one side, we have people who seem to feel that the existence of a document means something.<br />
1026. <br />
1027. I think that's a stupid view; it may have meaning when you have larger democratic projects and you need therefore written rules to justify actions -- and in that case a Code of Conduct is really a way to justify punishing someone, rather than prevention or education.  To those of you who think you need such a document in order to participate in a project -- I think you're acting like a bunch of spineless wimps.<br />
1028. <br />
1029. This isn't to say you should have to put up with abuse or toxic conduct.  But if you think a document creates a "safe space", you're smoking something funny.  Instead, look at the actual conduct of the project, and the actions of leadership.  A paper Code of Conduct isn't going to fix brokenness, and I have my doubts that it can prevent brokenness from occurring in the first place.<br />
1030. <br />
1031. If the leadership needs a CoC to correct toxic behavior, then the leadership of the project is busted.  Strong leadership leads by example, and takes the appropriate action to ensure that the communities that they lead are pleasant places to be.   (That's not necessarily the same as being conflict-free; much technical goodness comes about as a consequence of heartfelt debate, and developers can be just as passionate about the things they care about as anyone else.  Keeping the tone of such debate on topic and non-personal and professional is one of the signs of good leadership.)<br />
1032. <br />
1033. On the other side, are those who rail against such a document.  Are you so afraid of your own speech that you don't think you can agree to a document that basically says we are to treat each other respectfully?  The word I use for such people is "chickenshit".   If you can't or won't agree to be respectful towards others in the open source projects I lead, then I don't want your involvement.<br />
1034. <br />
1035. There's no doubt that there exists real abuse and intolerance in open source communities, and those who would cast aspersions on someone because of race, religion, physical attribute, or gender (or preference), are themselves slime, who really only underscore for everyone else their own ignorance and stupidity.  I have no tolerance for such bigotry, and I don't think anyone else should either.<br />
1036. <br />
1037. Don't misunderstand me; I'm not advocating for CoCs.  I think they are nearly worthless, and I resent the movement that demands that every project adopt one.  But I equally resent the strenuous opposition to their existence.  If a CoC does no good, it seems to me that it does no harm either.  So even if it is just a placebo effect, if it can avoid conflict and make a project more widely acceptable, then its worth having one, precisely because the cost of doing so is so low.<br />
1038. <br />
1039. Yes, this is "slacktivism".<br />
1040. <br />
1041. I've been taught that actions speak louder than words though.<br />
1042. <br />
1043. So today I'm stepping down.<br />
1044. <br />
1045. I'm retaining my BDFL of mangos, of course, so I'll still be around the nanomsg community, but I will be giving it far less of my energy.<br />
1046. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/3668235664676989390/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=3668235664676989390' title='8 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/3668235664676989390'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/3668235664676989390'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2016/01/stepping-down.html' title='Stepping Down'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>8</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-2789136103532913749</id><published>2015-12-11T10:46:00.002-08:00</published><updated>2015-12-11T10:52:21.420-08:00</updated><title type='text'>What Microsoft Can Do to Make Me Hate Windows a Little Less</title><content type='html'>Those who know me know that I have little love for Microsoft Windows. &amp;nbsp;The platform is a special snowflake, and coming from a Unix background (real &lt;a href=&quot;http://www.illumos.org/&quot;&gt;UNIX&lt;/a&gt;, not Linux, btw), every time I&#39;m faced with Windows I feel like I&#39;m in some alternate dimension where everything is a little strange and painful.&lt;br /&gt;
1047. <br />
1048. I have to deal with Windows because of applications.  My wife runs Quickbooks (which is one of the more chaotic and poorly designed bits of software I've run across), the kids have video games they like.  I've had to run it myself historically because some expense report site back at former employer AMD was only compatible with IE.  I also have a flight simulator for RC aircraft that only works in Windows (better to practice on the sim, no glue needed when you crash, just hit the reset button.)<br />
1049. <br />
1050. All of those are merely annoyances, and I keep Windows around on one of my computers for this reason.  It's not one I use primarily, nor one I carry with me when I travel.<br />
1051. <br />
1052. But I also have created and support software that runs on Windows, or that people want to use on Windows.  Software like <a href="http://www.nanomsg.org/">nanomsg</a>, <a href="http://github.com/gdamore/mangos">mangos</a>, <a href="http://github.com/gdamore/tcell">tcell</a>, etc.  This is stuff that supports other developers.  Its free and open software, and I make no money from any of it.<br />
1053. <br />
1054. Supporting that software is a pain on Windows, largely due to the fact that I don't have a Windows license to run Windows in a VM.  The only reason I'd buy such a license for my development laptop would be to support my free software development efforts.  Which would actually help and benefit the Windows ecosystem.<br />
1055. <br />
1056. I rely on <a href="http://www.appveyor.com/">AppVeyor</a> (which is an excellent service btw) to help me overcome my lack of a Windows instance on my development system.  This has allowed me to support some things pretty well, but the lack of an interactive command line means that some experiments are nigh impossible for me to try; others make me wait for the CI to build and test this, which takes a while.  Leading to lost time during the development cycle, all of which make me loathe working on the platform even more.<br />
1057. <br />
1058. Microsoft can fix this.  In their latest "incarnation", they are claiming to be open source friendly, and they've even made big strides here in supporting open source developers.  <a href="http://techcrunch.com/2014/11/12/microsoft-makes-visual-studio-free-for-small-teams/">Visual Studio is free</a> (as in beer).  Their <a href="https://code.visualstudio.com/download">latest code editor</a> is even open source.  The .Net framework itself is <a href="http://techcrunch.com/2014/11/12/microsoft-takes-net-open-source-and-cross-platform/">open source</a>.<br />
1059. <br />
1060. But the biggest barrier is the license for the platform itself.  I'm simply not going to run Windows on the bare metal -- I'm a Mac/UNIX guy and that is not going to change.  But I can and would be happier to occasionally run Windows to better support that platform in a VM, just like I do for <a href="http://www.illumos.org/">illumos</a> or Linux or FreeBSD.<br />
1061. <br />
1062. So, Microsoft, here's your chance to make me hate your platform a little less.  Give open source developers access to free Windows licenses; to avoid cannibalizing your business you could have license terms that only allow these free licenses to be used when Windows is run in a virtual machine for non-commercial purposes.  This is a small thing you could do, to extend your reach to a set of developers who've mostly abandoned you.<br />
1063. <br />
1064. (And Apple, there's a similar lesson there for you.  I'm a devoted MacOS X fan, but imagine how much wider your developer audience could be if you let people run MacOS X in a VM for non-commercial use?)<br />
1065. <br />
1066. In the meantime, if you use software I develop, please don&#39;t be surprised if you find that I treat Windows as a distinctly second class citizen. &amp;nbsp;After all, its no worse than how Microsoft has treated me as an open source developer.</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/2789136103532913749/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=2789136103532913749' title='4 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2789136103532913749'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2789136103532913749'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2015/12/what-microsoft-can-do-to-make-me-hate.html' title='What Microsoft Can Do to Make Me Hate Windows a Little Less'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>4</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-1493228101611806802</id><published>2015-12-08T16:41:00.000-08:00</published><updated>2015-12-08T16:42:33.526-08:00</updated><title type='text'>On Misunderstandings</title><content type='html'>Yesterday there was a flurry of activity on Twitter, and in retrospect, it seems that some have come away with interpretations of what I said that are other than what I intended. &amp;nbsp;Some of that misunderstanding is pretty unfortunate, so I&#39;d like to set the record straight on a couple of items now.&lt;br /&gt;
1067. <br />
1068. First off, let me begin by saying that this blog, and my Twitter account, are mine alone, and are used by me to express my opinions.  They represent neither <a href="http://www.illumos.org/">illumos</a> nor <a href="http://www.lucera.com/">Lucera</a>, nor anyone or anything else.<br />
1069. <br />
1070. Second, I have to apologize for it seems that I've come across as somehow advocating either against diversity (whether in the community or in the workplace) or in favor of toxicity.<br />
1071. <br />
1072. Nothing could be further from the truth.  I believe strongly in diversity and an inclusive environment, both for illumos, and in the work place.  I <a href="https://www.youtube.com/watch?v=302C7TaCsSY&feature=youtu.be&list=PLH8r-Scm3-2VmZhZ76tFPAhPOG0pvmjdA#t=13m30s">talked about this at illumos day last year</a> (see about 13:30 into the video, slides <a href="http://us-east.manta.joyent.com/illumos/public/events/illumosday2014/gdamore.pdf">here</a>), and I've also put my <a href="http://garrett.damore.org/2014/10/supporting-women-in-open-source.html">money where my mouth</a> is.  Clearly, it hasn't been enough, and I think we all can and should do better.  I'm interested in finding ways to increase the diversity in illumos in particular, and the industry in general.  Feel free to post your suggestions in the comments following this blog.<br />
1073. <br />
1074. Additionally, no, I don't believe that anyone should have to put up with "high performing toxic people".  The illumos community has appropriately censured people for toxic behavior in the past, and I was supportive of that action back then, and still am now.  Maintaining a comfortable work place and a comfortable community leads to increased personal satisfaction, and that leads to increased productivity.  Toxicity drives people away, and that flies in the face of the aforementioned desire for diversity (as well as the unstated ones for a growing and a healthy community.)<br />
1075. <br />
1076. Finally, I didn't mean to offend anyone.  If I've done so in my recent tweets, please be assured that this was not intentional, and I hope you'll accept my heartfelt apology.<br />
1077. <br />
1078. Thanks.</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/1493228101611806802/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=1493228101611806802' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/1493228101611806802'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/1493228101611806802'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2015/12/on-misunderstandings.html' title='On Misunderstandings'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-6042188200741696921</id><published>2015-10-19T04:29:00.001-07:00</published><updated>2015-10-19T04:29:36.762-07:00</updated><title type='text'>A Space Shooter in Curses</title><content type='html'>Some of you who follow me may know that I have recently built a pretty nifty framework for working with terminals. &amp;nbsp;ANSI, ASCII, VT100, Windows Console, etc. &amp;nbsp;Its called &lt;a href=&quot;http://github.com/gdamore/tcell&quot;&gt;Tcell&lt;/a&gt;, and located on github. &amp;nbsp;(Its a Go framework though.) &amp;nbsp;It offers many of the same features as curses, though it is most definitely not a clone of curses.&lt;br /&gt;
1079. <br />
1080. Anyway, I decided it should be possible to write a game in this framework, so I wrote one.<br />
1081. <br />
1082. I give you <a href="http://github.com/gdamore/proxima5">Escape From Proxima 5</a>, a 2D multi-axis scrolling space shooter written entirely in Go, designed to operate in your text terminal<br />
1083. <br />
1084. The game is fairly primordial, but there is a playable level complete with enemies and hazards.   It's actually reasonably difficult to get past just this first level.<br />
1085. <br />
1086. Mostly the idea here is that you can get a sense of what the game engine is capable of, and see Tcell in action.<br />
1087. <br />
1088. As part of this, I wrote a pretty complete 2D game engine.  Its got rich sprite management with collision detection, palettes, an events subsystem, scrolling maps, and support for keyboards and mice.  Its also got pretty nice extensibility as assets are defined in YAML files that are converted and compiled into the program.  (I guess an asset editor needs to be written. :-)<br />
1089. <br />
1090. The code is <a href="https://github.com/gdamore/proxima5/blob/master/LICENSE">Apache 2 licensed</a>, so feel free to borrow bits for your own projects.  I'd love to hear about it.<br />
1091. <br />
1092. Anyway, I thought I'd post this here.  I made two videos.  The longer one, at about 3:30, shows most of the features of the game, animated sprites, some nice explosions, gravity effects, beam field effects, etc.<br />
1093. <br />
1094. <div class="separator" style="clear: both; text-align: center;">
1095. <iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/jNxKTCmY_bQ/0.jpg" src="https://www.youtube.com/embed/jNxKTCmY_bQ?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
1096. <br />
1097. <br />
1098. The second video shows what this looks like on less rich terminals -- say a VT100 with only 7-bit ASCII characters available.  The richer your locale, the nicer it will look.  But it falls down as gracefully as one can expect.<br />
1099. <br />
1100. <div class="separator" style="clear: both; text-align: center;">
1101. <iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/DiOPBBM7-Xc/0.jpg" src="https://www.youtube.com/embed/DiOPBBM7-Xc?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
1102. <br />
1103. Btw, this framework is now basically design complete, so it should be super easy to product a lot of simples kinds of games -- for example a clone of Missile Command or Space Invaders should be doable in an afternoon. &amp;nbsp; What makes this game a little bigger is the number if different kinds of objects and object interactions we can have.</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/6042188200741696921/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=6042188200741696921' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/6042188200741696921'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/6042188200741696921'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2015/10/a-space-shooter-in-curses.html' title='A Space Shooter in Curses'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://img.youtube.com/vi/jNxKTCmY_bQ/default.jpg" height="72" width="72"/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-2568373107269807122</id><published>2015-10-05T00:02:00.001-07:00</published><updated>2015-10-05T00:02:07.569-07:00</updated><title type='text'>Fun with terminals, character sets, Unicode, and Go</title><content type='html'>As part of my recent work on &lt;a href=&quot;http://github.com/gdamore/tcell&quot;&gt;Tcell&lt;/a&gt;, I&#39;ve recently added some pretty cool functionality for folks who want to have applications that can work reasonably in many different locales.&lt;br /&gt;
1104. <br />
1105. For example, if your terminal is running in UTF-8, you can access a <i>huge</i> repertoire of glyphs / characters.<br />
1106. <br />
1107. But if you're running in a non-UTF-8 terminal, such as an older ISO 8859-1 (Latin1) or KOI8-R (Russian) locale, you might have problems.  Your terminal won't be able to display UTF-8, and your key strokes will probably be reported to the application using some 8-bit variant that is incompatible with UTF-8.  (For ASCII characters, everything will work, but if you want to enter a different character, like Я (Russian for "ya"), you're going to have difficulties.<br />
1108. <br />
1109. If you work on the console of your operating system, you probably have somewhere around 220 characters to play with.  You're going to miss some of those glyphs.<br />
1110. <br />
1111. <a href="http://golang.org/">Go</a> of course works with UTF-8 natively.  Which is just awesome.<br />
1112. <br />
1113. Until you have to work in one of these legacy environments.   And some of the environments are not precisely "legacy".  (<a href="http://gb18030.com/">GB18030</a> has the same repertoire as UTF-8, but uses a different encoding scheme and is legally mandatory within China.)<br />
1114. <br />
1115. If you use Tcell for your application's user interface, this is now "fixed".<br />
1116. <br />
1117. Tcell will attempt to convert to characters that the user's terminal understands on output, provided the user's environment variables are set properly ($LANG, $LC_ALL, $LC_CTYPE, per POSIX).  It will also convert the user's key strokes from your native locale to UTF-8.  This means that YOU, the application developer, can just worry about UTF-8, and skip the rest.  (Unless you want to add new Encodings, which is entirely possible.)<br />
1118. <br />
1119. Tcell even goes further.<br />
1120. <br />
1121. It will use the alternate character set (ACS) to convert Unicode drawing characters to the characters supported by the terminal, if they exist -- or to reasonable ASCII fallbacks if they don't.  (Just like <a href="https://www.gnu.org/software/ncurses/">ncurses</a>!)<br />
1122. <br />
1123. It will also cope with both East Asian full-width (or ambiguous width) characters, and even properly handles combining characters.  (If your terminal supports it, these are rendered properly on the terminal.  If it doesn't, Tcell makes a concerted effort to make a best attempt at rendering -- preserving layout and presenting the primary character even if the combining character cannot be rendered.)<br />
1124. <br />
1125. The Unicode (and non-Unicode translation) handling capabilities in Tcell far exceed any other terminal handling package I'm aware of.<br />
1126. <br />
1127. Here are some interesting screen caps, taken on a Mac using the provided unicode.go test program.<br />
1128. <br />
1129. First the UTF-8.  Note the Arabic, the correct spacing of the Chinese glyphs, and the correct rendering of Combining characters.  (Note also that emoji are reported as width one, instead of two, and so take up more space than they should.  This is a font bug on my system -- Unicode says these are Narrow characters.)<br />
1130. <div class="separator" style="clear: both; text-align: center;">
1131. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhp5PKGzWHSnykSLgGLLk4Aj-VNdcXpXuf19AfTIgHdagsjej0qSdQ87vP9jDDjUu6k_sWx03gSCpEcM0eG6NJdrPsWlqyQBs8iZuv0Gscf7xfHTlWIaeIUmuHicEz27XV4nEAV1vTMUhk/s1600/Screen+Shot+2015-10-04+at+11.39.27+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhp5PKGzWHSnykSLgGLLk4Aj-VNdcXpXuf19AfTIgHdagsjej0qSdQ87vP9jDDjUu6k_sWx03gSCpEcM0eG6NJdrPsWlqyQBs8iZuv0Gscf7xfHTlWIaeIUmuHicEz27XV4nEAV1vTMUhk/s320/Screen+Shot+2015-10-04+at+11.39.27+PM.png" width="320" /></a></div>
1132. <div class="separator" style="clear: both; text-align: left;">
1133. Then we run in ISO8859-1 (Latin 1).  Here you can see the accented character available in the Icelandic word, and some terminal specific replacements have been made for the drawing glyphs.  ISO 8859-1 lacks most of the unusual or Asian glyphs, and so those are rendered as "?".  This is done by Tcell -- the terminal <i>never sees the raw Unicode/UTF-8</i>.  That's important, since sending the raw UTF-8 could cause my terminal to do bad things.</div>
1134. <div class="separator" style="clear: both; text-align: left;">
1135. <br /></div>
1136. <div class="separator" style="clear: both; text-align: left;">
1137. Note also that the widths are properly handled, so that even though we cannot display the combining characters, nor the full-width Chinese characters, the widths are correct -- 1 cell is taken for the combining character combinations, and 2 cells are taken by the full width Chinese characters.</div>
1138. <br />
1139. <div class="separator" style="clear: both; text-align: center;">
1140. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdYHbjz8NGgh3H95eaixbNDLOi5yjbB_T1nDMTea3gqr4ZDqdycBJMOCJ4mG4HrNT31owjSY-xB7k3fcAvuOp-W00ykr4bUSt5lcTVI7G7ZbYUAqZ2SHfspfDAF8CreimBUtaxHSt85mY/s1600/Screen+Shot+2015-10-04+at+11.39.56+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdYHbjz8NGgh3H95eaixbNDLOi5yjbB_T1nDMTea3gqr4ZDqdycBJMOCJ4mG4HrNT31owjSY-xB7k3fcAvuOp-W00ykr4bUSt5lcTVI7G7ZbYUAqZ2SHfspfDAF8CreimBUtaxHSt85mY/s320/Screen+Shot+2015-10-04+at+11.39.56+PM.png" width="320" /></a></div>
1141. <div class="separator" style="clear: both; text-align: left;">
1142. Then we show off legacy Russian (KOI8-R):   Here you can see Cyrillic is rendered properly, as well as the basic ASCII and the alternate (ACS) drawing characters (mostly), while the rest are filled with place holder ?'s.</div>
1143. <br />
1144. <div class="separator" style="clear: both; text-align: center;">
1145. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZkZN_cBU2lDzcjtKBXZDRjurXxsLN499kzgp41zw3b79wlfbGmX_7CdxqOf7JdhSG4MacUkPGy7Zim3Q-DDxOWvBSg-K9VbMrIeIBuMBT6vY2KmasGXE7mBIvf3vRErGtFW351CGm3i4/s1600/Screen+Shot+2015-10-04+at+11.40.30+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZkZN_cBU2lDzcjtKBXZDRjurXxsLN499kzgp41zw3b79wlfbGmX_7CdxqOf7JdhSG4MacUkPGy7Zim3Q-DDxOWvBSg-K9VbMrIeIBuMBT6vY2KmasGXE7mBIvf3vRErGtFW351CGm3i4/s320/Screen+Shot+2015-10-04+at+11.40.30+PM.png" width="320" /></a></div>
1146. <div class="separator" style="clear: both; text-align: left;">
1147. And, for those of you in mainland China, here's GB18030:  Its somewhat amusing that the system font seems to not be able to cope with the combining enclosure here.  Again, this is a font deficiency in the system.</div>
1148. <br />
1149. <div class="separator" style="clear: both; text-align: center;">
1150. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3z_-BhYfnOPE4GDCoal7Csnl702sJKVGI6Z2kQA2m3Pdvf-gtyEwkdslHl_mIFY8WnAfu00ABL0gLw3bxWcNMeLJ51VczYZX_MhjpKVeDzvhPUOE1L4u_NvFagptn-FPPUT2MrYdAFMM/s1600/Screen+Shot+2015-10-04+at+11.40.55+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3z_-BhYfnOPE4GDCoal7Csnl702sJKVGI6Z2kQA2m3Pdvf-gtyEwkdslHl_mIFY8WnAfu00ABL0gLw3bxWcNMeLJ51VczYZX_MhjpKVeDzvhPUOE1L4u_NvFagptn-FPPUT2MrYdAFMM/s320/Screen+Shot+2015-10-04+at+11.40.55+PM.png" width="320" /></a></div>
1151. <div class="separator" style="clear: both; text-align: center;">
1152. <br /></div>
1153. <div class="separator" style="clear: both; text-align: left;">
1154. As you can see, we have a lot of rendering options.  Input is filtered and converted too.  Unfortunately, the mouse test program I use to verify this doesn't really show this (since you can't see what I typed), but the Right Thing happens on input too.</div>
1155. <div class="separator" style="clear: both; text-align: left;">
1156. <br /></div>
1157. <div class="separator" style="clear: both; text-align: left;">
1158. Btw, those of you looking for mouse usage in your terminal should be very very happy with Tcell.  As far as I can tell, Tcell offers improved mouse handling on stock XTerm over every other terminal package.  This includes live mouse reporting, click-drag reporting, etc.   Here's what the test program looks like on my system, after I've click-dragged to create a few boxes:</div>
1159. <div class="separator" style="clear: both; text-align: left;">
1160. <br /></div>
1161. <div class="separator" style="clear: both; text-align: center;">
1162. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiO5rek_BkJCsXk8EnHtH0YXvV93zTJw9OU2IHbZxpFEd2ThJ1RIXjpNwbySxaoOWyk_wIa1FWOioaVxfIp5V5s2-7q7i2UhqfF126XC71BgoKOaTNJrQBYI3j6DQehBNQZn8Qd6TcKjSk/s1600/Screen+Shot+2015-10-04+at+11.47.13+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiO5rek_BkJCsXk8EnHtH0YXvV93zTJw9OU2IHbZxpFEd2ThJ1RIXjpNwbySxaoOWyk_wIa1FWOioaVxfIp5V5s2-7q7i2UhqfF126XC71BgoKOaTNJrQBYI3j6DQehBNQZn8Qd6TcKjSk/s320/Screen+Shot+2015-10-04+at+11.47.13+PM.png" width="320" /></a></div>
1163. <div class="separator" style="clear: both; text-align: left;">
1164. <br /></div>
1165. <div class="separator" style="clear: both; text-align: left;">
1166. <br /></div>
1167. <div class="separator" style="clear: both; text-align: left;">
1168. I'm super tempted to put all this together to write an old DOS-style game.  I think Tcell has everything necessary here to be used as the basis for some really cool terminal hacks.</div>
1169. <div class="separator" style="clear: both; text-align: left;">
1170. <br /></div>
1171. <div class="separator" style="clear: both; text-align: left;">
1172. Give it a whirl if you like, and let me know what you think.</div>
1173. <div class="separator" style="clear: both; text-align: left;">
1174. <br /></div>
1175. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/2568373107269807122/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=2568373107269807122' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2568373107269807122'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/2568373107269807122'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2015/10/fun-with-terminals-character-sets.html' title='Fun with terminals, character sets, Unicode, and Go'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhp5PKGzWHSnykSLgGLLk4Aj-VNdcXpXuf19AfTIgHdagsjej0qSdQ87vP9jDDjUu6k_sWx03gSCpEcM0eG6NJdrPsWlqyQBs8iZuv0Gscf7xfHTlWIaeIUmuHicEz27XV4nEAV1vTMUhk/s72-c/Screen+Shot+2015-10-04+at+11.39.27+PM.png" height="72" width="72"/><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-9177932589493626125</id><published>2015-10-02T22:25:00.000-07:00</published><updated>2015-10-02T22:25:06.667-07:00</updated><title type='text'>Tcell  - Terminal functionality for Pure Go apps</title><content type='html'>Introducing &lt;a href=&quot;http://github.com/gdamore/tcell&quot;&gt;Tcell&lt;/a&gt; &amp;nbsp;- Terminal functionality for Pure Go apps&lt;br /&gt;
1176. <br />
1177. As part of the work I've done on <a href="http://github.com/gdamore/govisor">govisor</a>, I had a desire for rich terminal functionality so that I could build a portable curses-style management application.<br />
1178. <br />
1179. This turned out to be both easier and harder than I thought.<br />
1180. <br />
1181. Easier, because there was an implementation to start from -- <a href="http://github.com/nsf/termbox-go">termbox-go</a>, but harder because that version wasn't portable to the <a href="http://www.illumos.org/">OS I cared most about</a>, and fell short in some ways that I felt were important.  (Specifically, reduced functionality for mice, colors, and Unicode.)<br />
1182. <br />
1183. This led me to create my own library; I've made some very very different design choices.  These design choices have let me both support more platforms (pretty much all POSIX platforms and Windows are included), increase support for a much richer set of terminals and terminal capabilities, etc.<br />
1184. <br />
1185. The work is called "<a href="http://github.com/gdamore/tcell">Tcell</a>" (for terminal cells, which is the unit we operate on -- if you don't like the name ... well I'm open to suggestions.  I freely admit that I suck at naming -- and its widely <a href="http://martinfowler.com/bliki/TwoHardThings.html">acknowledged</a> that naming is one of the "hard" problems in computer science.)<br />
1186. <br />
1187. As part of this work, I've implemented a full Terminfo string parser/expander in Go.  This isn't quite as trivial as you might think, since the parameterized strings actually have a kind of stack based "<a href="http://invisible-island.net/ncurses/man/terminfo.5.html#h3-Parameterized-Strings">minilanguage</a>", including conditionals, in them.<br />
1188. <br />
1189. Furthermore, I've wound up exploring lots more about mice, and terminals, and the most common emulators.  As a result, I think I have created a framework that can support very rich mouse reporting, key events, colors, and Unicode.  Of course, the functionality will vary somewhat by platform and terminal (your <a href="https://en.wikipedia.org/wiki/VT52#/media/File:Terminal-dec-vt52.jpg">vt52</a> isn't going to support rich colors, Unicode, or even a mouse, after all.  But Xterm can, as can most modern emulators.)<br />
1190. <br />
1191. This work would be a nice basis for portable readline implementations, getpassphrase, etc.  Or a full up curses implementation.  It does support a compatibility mode for termbox, and most termbox applications work with only changing the import line.   I still need to change govisor, and the topsl panels library, to use tcell, but part of that work will be including much more capable mouse reporting, which tcell now facilitates.<br />
1192. <br />
1193. Admittedly, if you cannot tolerate cgo in any of your work -- this isn&#39;t for you -- yet. &amp;nbsp;I will probably start adding non-cgo implementations for particular ports such as Linux, Darwin, and FreeBSD. &amp;nbsp;But for other platforms there will always be cgo as a fallback. &amp;nbsp;Sadly, unless the standard Go library evolves to add sufficient richness that all of the termios functionality are available natively on all platforms (which is a real possibility), cgo will remain necessary. &amp;nbsp;(Note that Windows platform support does &lt;i&gt;not&lt;/i&gt; use CGO.)</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/9177932589493626125/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=9177932589493626125' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/9177932589493626125'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/9177932589493626125'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2015/10/tcell-terminal-functionality-for-pure.html' title='Tcell  - Terminal functionality for Pure Go apps'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-4853466830148344524</id><published>2015-09-22T12:05:00.001-07:00</published><updated>2015-09-22T12:05:49.293-07:00</updated><title type='text'>On Go, Portability, and System Interfaces</title><content type='html'>I&#39;ve been noticing more and more lately that we have a plethora of libraries and programs written for Go, which don&#39;t work on one platform or another. &amp;nbsp;The root cause of these is often the use of direct system call coding to system calls such as ioctl(). &amp;nbsp;On some platforms (illumos/solaris!) there is no such system call.&lt;br /&gt;
1194. <br />
1195. <h3>
1196. The Problems</h3>
1197. <br />
1198. But this underscores a far far worse problem, that has become common (mal)-practice in the <a href="http://www.golang.org/">Go</a> community.  That is, the coding of system calls directly into high level libraries and even application programs.  For example, it isn't uncommon to see something like this (taken from <a href="http://github.com/nsf/termbox-go">termbox-go</a>):<br />
1199. <blockquote class="tr_bq">
1200. <span style="font-family: Verdana, sans-serif; font-size: x-small;"><span style="color: #cc0000;">func</span> tcsetattr(fd uintptr, termios *syscall_Termios) <span style="color: #cc0000;">error</span> {</span><br />
1201. <span style="font-family: Verdana, sans-serif; font-size: x-small;">        r, _, e := syscall.Syscall(syscall.SYS_IOCTL,</span><br />
1202. <span style="font-family: Verdana, sans-serif; font-size: x-small;">                fd, uintptr(syscall_TCSETS), uintptr(unsafe.Pointer(termios)))</span><br />
1203. <span style="font-family: Verdana, sans-serif; font-size: x-small;">        if r != 0 {</span><br />
1204. <span style="font-family: Verdana, sans-serif; font-size: x-small;">                <span style="color: #cc0000;">return</span> os.NewSyscallError("<span style="color: blue;">SYS_IOCTL</span>", e)</span><br />
1205. <span style="font-family: Verdana, sans-serif; font-size: x-small;">        }</span><br />
1206. <span style="font-family: Verdana, sans-serif; font-size: x-small;">        <span style="color: #cc0000;">return nil</span></span><br />
1207. <span style="font-family: Verdana, sans-serif; font-size: x-small;">}</span></blockquote>
1208. <br />
1209. This has quite a few problems with it.<br />
1210. <ol>
1211. <li>It's not platform portable.  This function depends on a specific implementation of <a href="http://pubs.opengroup.org/onlinepubs/009695399/functions/tcsetattr.html">tcsetattr</a>() that is done in terms of specific ioctl()s.  For example, TCSETS may be used on one platform, but on others TIOCSETA might be used.</li>
1212. <li>It's not Go portable, since SYS_IOCTL isn't implemented on platforms like <a href="http://www.illumos.org/">illumos</a>, even though as a <a href="http://pubs.opengroup.org/onlinepubs/9699919799/">POSIX</a> system we do have a working tcsetattr(). </li>
1213. <li>The code is actually pretty unreadable, and somewhat challenging to write the first time correctly.</li>
1214. <li>The code uses unsafe.Pointer(), which is clearly something we ought to avoid.</li>
1215. <li>On some platforms, the details of the ioctls are subject to change, so that the coding above is actually fragile.  (In illumos & Solaris system call interfaces are "undocumented", and one must use the C library to access system services.  This is our "stable API boundary".  This is somewhat different from Linux practice; the reasons for this difference is both historical and related to the fact that Linux delivers only a kernel, while illumos delivers a system that includes both the kernel and core libraries.)</li>
1216. </ol>
1217. <b><i>How did we wind up in this ugly situation?</i></b><br />
1218. <br />
1219. The problem I believe stems from some misconceptions, and some historical precedents in the Go community.   First the Go community has long touted static linking as one of its significant advantages.  However, I believe this has been taken too far.<br />
1220. <br />
1221. Why is static linking beneficial?  The obvious (to me at any rate) reason is to avoid the dependency nightmares and breakage that occurs with other systems where many dynamic libraries are brought together.  For example, if A depends directly on both B and C, and B depends on C, but some future version of B depends on a newer version of C that is incompatible with the version of C that A was using, then we cannot update A to use the new B.  And when the system components are shared across the entire system, the web of dependencies gets to be so challenging that managing these dependencies in real environments can become a full time job, consuming an entire engineering salary.<br />
1222. <br />
1223. You can get into surprising results where upgrading one library can cause unexpected failures in some other application.  So the desire to avoid this kind of breakage is to encode the entire binary together, in a single stand-alone executable, so that we need never have a fear as to whether our application will work in the future or not.  As I will show, we've not really achieved this with 100% statically linked executables in Go, though I'll grant that we have greatly reduced the risk.<br />
1224. <br />
1225. This is truly necessary because much of the open source ecosystem has no idea about interface stability nor versioning interfaces.  This is gradually changing, such that we now have ideas like semver coming around as if they are somehow new and great ideas.  The reality is that commercial operating system vendors have understood the importance of stable API boundaries for a very very long time.  Some, like Sun, even made legally binding promises around the stability of their interfaces.  However, in each of these cases, the boundary has to a greater or lesser extent been at the discretion of the vendor.<br />
1226. <br />
1227. Until we consider standards such as POSIX 1003.1.  Some mistakenly believe that POSIX defines system calls.  It does not.  It defines a C function call interface.  The expectation is that many of these interfaces have 1:1 mappings with system calls, but the details of those system calls are completely unspecified by POSIX.<br />
1228. <br />
1229. Basically, the Go folks want to minimize external dependencies and the web of failure that can lead to.  Fixing that is a goal I heartily agree with.  However, we cannot eliminate our dependency on the platform.  And using system calls directly is actually worse, because it moves our dependency from something that is stable and defined by standards bodies, to an interface that is undocumented, not portable, and may change at any time.<br />
1230. <br />
1231. If you're not willing to have a dynamic link dependency on the C library, why would you be willing to have a dependency on the operating system kernel?  In fact, the former is far safer than the latter! (And on Solaris, you don't have a choice -- the Go compiler always links against the system libraries.)<br />
1232. <br />
1233. <b><i>Harmful results that occur with static linking</i></b><br />
1234. <br />
1235. If the application depends on a library that has a critical security update, it becomes necessary to recompile the application.  If you have a low level library such as a TLS or HTTP client, and a security fix for a TLS bug is necessary (<i>and we've never ever ever had any bugs in TLS or SSL implementation, right?</i>), this could mean recompiling a very large body of software to be sure you've closed the gaps.<br />
1236. With statically linked programs, even knowing which applications need to be updated can be difficult or impossible.  They defy the most easy kinds of inspection, using tools like ldd or otool to see what they are built on top of.<br />
1237. <br />
1238. What is also tragic, is that static executables wind up encoding the details of the kernel system call interface into the binary.  On some systems this isn't a big deal because they have a stable system call interface.  (Linux mostly has this -- although glibc still has to cope with quite a few differences here by handling ENOSYS, and don't even get me started on systemd related changes.)  But on systems like Solaris and illumos, we've historically considered those details a private implementation detail between libc and kernel.  And to prevent applications from abusing this, we don't even deliver a static libc.  This gives us the freedom to change the kernel/userland interface fairly freely, without affecting applications.<br />
1239. <br />
1240. When you consider standards specifications like POSIX or X/OPEN, this approach makes a lot of sense.  They standardize the C function call interface, and leave the kernel implementation up to the implementor.<br />
1241. <br />
1242. But statically linked Go programs break this, badly.  If that kernel interface changes, we can wind up breaking all of the Go programs that use it, although "correct" programs that only use libc will continue to work fine.<br />
1243. <br />
1244. <b><i>The elephant in the room (licensing)</i></b><br />
1245. <br />
1246. The other problem with static linking is that it can create a license condition that is very undesirable.  For example, glibc is LGPL.  That means that per the terms of the LGPL it must be possible to relink against a different glibc, if you link statically.<br />
1247. <br />
1248. Go programs avoid this by not including any of the C library statically. Even when cgo is used, the system libraries are linked dynamically.  (This is usually the C library, but can include things like a pthreads library or other base system libraries.)<br />
1249. <br />
1250. In terms of the system, the primary practice for Go programmers has been to use licenses like MIT, BSD, or Apache, that are permissive enough that static linking of 3rd party Go libraries is usually not a problem.  I suppose that this is a great benefit in that it will serve to help prevent GPL and LGPL code from infecting the bulk of the corpus of Go software.<br />
1251. <br />
1252. <h3>
1253. The Solutions</h3>
1254. <br />
1255. The solution here is rather straightforward.<br />
1256. <br />
1257. First, we should not eschew use of the C library, or other libraries that are part of the standard system image.  I'm talking about things like libm, libc, and for those that have them, libpthread, libnsl, libsocket.  Basically the standard libraries that every non-trivial program has to include.  On most platforms this is just libc.  If recoded to use the system's tcsetattr (which is defined to exist by POSIX), the above function looks like this:<br />
1258. <blockquote class="tr_bq">
1259. <span style="font-family: Verdana, sans-serif; font-size: x-small;"><span style="color: #6aa84f;">// include <termios.h></span><br /><span style="color: #cc0000;">import</span> "<span style="color: blue;">C</span>"<br /><span style="color: #cc0000;">import</span> "<span style="color: blue;">os</span>"</span> </blockquote>
1260. <blockquote class="tr_bq">
1261. <span style="font-family: Verdana, sans-serif; font-size: x-small;"><span style="color: #cc0000;">func</span> tcsetattr(f *os.File, termios *C.struct_termios) <span style="color: #cc0000;">error</span> {<br />        _, e := C.tcsetattr(C.int(f.Fd(), C.TCSANOW, termios)<br />        <span style="color: #cc0000;">return</span> e<br />}</span></blockquote>
1262. The above implementation will cause your library or program to dynamically link against and use the standard C library on the platform.  And it works on all POSIX systems everywhere and because it uses a stable documented standard API, it is pretty much immune to breakage from changes elsewhere in the system.  (At least any change that broke this implementation would also break so many other things that the platform would be unusable.  Generally we can usually trust people who make the operating system kernel and C library to not screw things up too badly.)<br />
1263. <br />
1264. What would be even better, and cleaner, would be to abstract that interface above behind some Go code, converting between a Go struct and the C struct as needed, just as is done in much of the rest of the Go runtime.  The logical place to do this would be in the standard Go system libraries.  I'd argue rather strongly that core services like termio handling ought to be made available to Go developers in the standard system libraries that are part of Go, or perhaps more appropriately, with the <a href="http://golang.org/x/sys/unix">golang.org/x/sys/unix</a> repository.<br />
1265. <br />
1266. In any event, if you're a Go programmer, please consider NOT directly calling syscall interfaces, but instead using higher level interfaces, and when those aren't already provided in Go, don't be afraid to use cgo to access standard functions in the C library.  Its far far better for everyone that you do this, than that you code to low level system calls.<br />
1267. &lt;br /&gt;</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/4853466830148344524/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=4853466830148344524' title='5 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4853466830148344524'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/4853466830148344524'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2015/09/on-go-portability-and-system-interfaces.html' title='On Go, Portability, and System Interfaces'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>5</thr:total></entry><entry><id>tag:blogger.com,1999:blog-7528831701633643336.post-1719934006778171557</id><published>2015-09-20T09:12:00.000-07:00</published><updated>2015-09-20T09:12:30.638-07:00</updated><title type='text'>Announcing govisor 1.0</title><content type='html'>I&#39;m happy to announce that I feel I&#39;ve wrapped up &lt;a href=&quot;https://github.com/gdamore/govisor&quot;&gt;Govisor&lt;/a&gt; to a point where its ready for public consumption.&lt;br /&gt;
1268. <br />
1269. Govisor is a service similar to <a href="http://supervisord.org/">supervisord</a>, in that it can be used to manage a bunch of processes.  However, it is much richer in that it understands process dependencies, conflicts, and also offers capabilities for self-healing, and consolidated log management.<br />
1270. <br />
1271. It runs as an ordinary user process, and while it has some things in common with programs like init, upstart, and Solaris SMF, it is not a replacement for any of those things.  Instead think of this is a portable way to manage a group of processes without requiring root.  In my case I wanted something that could manage a tree of microservices that was deployable by normal users.  Govisor is my answer to that problem.<br />
1272. <br />
1273. Govisor is also written entirely in Go, and is embeddable in other projects.  The REST server interface uses a stock http.ServeHTTP interface, so it can be used with various middleware or frameworks like the Gorilla toolkit.<br />
1274. <br />
1275. Services can be implemented as processes, or in native Go.<br />
1276. <br />
1277. Govisor also comes with a nice terminal oriented user interface (I'd welcome a JavaScript based UI, but I don't write JS myself).  Obligatory screen shots below.<br />
1278. <br />
1279. <div class="separator" style="clear: both; text-align: center;">
1280. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5Yi2E4CtTKD0qOARgNmWHxySRIa8xMkJCgxrPiQoTn7QZISmnuzqjtCv3hIv6pmUa1cTjAv_2F2ALuLJAIqyCaiKgWWemVJsGljJqhKEpT3WwMpvGl64ZEcA6iO8MikCCw7Fs_gJ9FXE/s1600/Screen+Shot+2015-09-20+at+9.08.41+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5Yi2E4CtTKD0qOARgNmWHxySRIa8xMkJCgxrPiQoTn7QZISmnuzqjtCv3hIv6pmUa1cTjAv_2F2ALuLJAIqyCaiKgWWemVJsGljJqhKEpT3WwMpvGl64ZEcA6iO8MikCCw7Fs_gJ9FXE/s320/Screen+Shot+2015-09-20+at+9.08.41+AM.png" width="320" /></a></div>
1281. <br />
1282. <div class="separator" style="clear: both; text-align: center;">
1283. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2tJwqxugvJvjYhB8Pm2BKc8zlOXvYZoXQC_IZduiFoaec4tueF9UuznAj4TvTiPL8ddV7_2A7uf_VCdgPArNVkYTpdglAlcTiKEqaUw9ngEmfQr_R-MLV4790CQVSaZlC0JnpMxpQ6uU/s1600/Screen+Shot+2015-09-20+at+9.08.54+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2tJwqxugvJvjYhB8Pm2BKc8zlOXvYZoXQC_IZduiFoaec4tueF9UuznAj4TvTiPL8ddV7_2A7uf_VCdgPArNVkYTpdglAlcTiKEqaUw9ngEmfQr_R-MLV4790CQVSaZlC0JnpMxpQ6uU/s320/Screen+Shot+2015-09-20+at+9.08.54+AM.png" width="320" /></a></div>
1284. <br />
1285. <div class="separator" style="clear: both; text-align: center;">
1286. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQY-s7VFAp87r8OM8eSQpsxMqOa22Meg7Bjlhk37dkmV3LEdWIms6w5KDZxY5yYyhL2STyZVQcxCJSrD5_JZNKGfzFYlfyytFObaTo9MCavErNg8MQ4ZN1L8LniBmZ_BWoRbBgjfUwIqM/s1600/Screen+Shot+2015-09-20+at+9.09.11+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQY-s7VFAp87r8OM8eSQpsxMqOa22Meg7Bjlhk37dkmV3LEdWIms6w5KDZxY5yYyhL2STyZVQcxCJSrD5_JZNKGfzFYlfyytFObaTo9MCavErNg8MQ4ZN1L8LniBmZ_BWoRbBgjfUwIqM/s320/Screen+Shot+2015-09-20+at+9.09.11+AM.png" width="320" /></a></div>
1287. <br />
1288. Which actually brings up the topic of "<a href="https://github.com/gdamore/topsl">tops'l</a>"  (which is a contraction of top-sail, often used in sailing).  Govisor depends the package tops -- "terminal oriented panels support library"), which provides a mid-level API for interacting with terminals.   I created topsl specifically to help me with creating the govisor client application.  I do consider topsl ready for use by Govisor, but I advise caution if you want to use it your own programs -- its really young and there are probably breaking changes looming in its future.<br />
1289. <br />
1290. The documentation is a bit thin on the ground, but if you want to help or have questions, just let me know! &amp;nbsp;In the meantime, enjoy!</content><link rel='replies' type='application/atom+xml' href='http://garrett.damore.org/feeds/1719934006778171557/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=7528831701633643336&amp;postID=1719934006778171557' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/1719934006778171557'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/7528831701633643336/posts/default/1719934006778171557'/><link rel='alternate' type='text/html' href='http://garrett.damore.org/2015/09/announcing-govisor-10.html' title='Announcing govisor 1.0'/><author><name>Garrett D&#39;Amore</name><uri>http://www.blogger.com/profile/12381273059232810608</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5Yi2E4CtTKD0qOARgNmWHxySRIa8xMkJCgxrPiQoTn7QZISmnuzqjtCv3hIv6pmUa1cTjAv_2F2ALuLJAIqyCaiKgWWemVJsGljJqhKEpT3WwMpvGl64ZEcA6iO8MikCCw7Fs_gJ9FXE/s72-c/Screen+Shot+2015-09-20+at+9.08.41+AM.png" height="72" width="72"/><thr:total>0</thr:total></entry></feed>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

1. Download the "valid Atom 1.0" banner.

2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

3. Add this HTML to your page (change the image src attribute if necessary):

   <a href="http://www.feedvalidator.org/check.cgi?url=http%3A//gdamore.blogspot.com/atom.xml"><img src="valid-atom.png" alt="[Valid Atom 1.0]" title="Validate my Atom 1.0 feed" /></a>

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//gdamore.blogspot.com/atom.xml

Home · About · News · Docs · Terms