FEED Validator

for Atom and RSS and KML

Congratulations!

This is a valid RSS feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

line 73, column 401: description should not contain relative URL references: /images/ChristianMiller_snotbot_intel-B22-500x333.jpg (18 occurrences) [help]

... c and Atmospheric Administration.</p>]]></description>
                                             ^

Source: http://www.xoltar.org/rss.xml

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:dc="http://purl.org/dc/elements/1.1/">
<channel>
<title>Xoltar</title>
<link>http://www.xoltar.org</link>
<description><![CDATA[A blog by Bryn Keller about simplicity and clarity in software and in life]]></description>
<atom:link href="http://www.xoltar.org/rss.xml" rel="self"
type="application/rss+xml" />
<lastBuildDate>Thu, 26 Apr 2018 00:00:00 UT</lastBuildDate>
<item>
<title>Machine learning, drones, and whales: A great combination!</title>
<link>http://www.xoltar.org/posts/2018-04-26-whale-expedition/index.html</link>
<description><![CDATA[Last June, a simple question changed my life: “Hey Bryn, what do you know about whales?” Of course like most people, my answer was “Not much,” but that marked the beginning of an important project to help track the health of whale populations by using machine learning to analyze video from drones. <a href="https://parley.tv">Parley for the Oceans</a> introduced me and my colleagues Ted Willke and Javier Turek to Dr. Iain Kerr of <a href="https://www.whale.org">Ocean Alliance</a>, and we started talking about how machine learning could help make marine biologists’ lives easier and help protect the whales, using the video from Dr. Kerr’s <a href="https://shop.whale.org/pages/snotbot">SnotBot</a> drones.
<div class="figure">
<img src="/images/ChristianMiller_snotbot_intel-B22-500x333.jpg" alt="SnotBot drone with Petri dishes, courtesy of Christian Miller" style="padding-left:3em" />
SnotBot drone with Petri dishes, courtesy of Christian Miller
</div>

<h1 id="snotbot">SnotBot</h1>
Dr. Kerr started the SnotBot program because in the not-so-distant past, when people wanted to understand the health of a whale, or get its DNA, the only way to do this was to shoot it with a crossbow with a specially prepared bolt with a string on it, that would only go a couple of inches into the (remember, bus-size) body. The bolt would then be reeled in and the sample could be extracted from it. Iain realized there was a better way, that didn’t involve crossbows or Zodiac inflatable rafts: put a Petri dish on a drone, and fly the drone through the blow that whales exhale when they come to the surface. It turns out there are all kinds of interesting things in that exhaled breath condensate (i.e. snot), like DNA, hormones, and microbiomes. Much less stress on the whale, and easier on the humans, too.
<h1 id="drones-and-cameras">Drones and Cameras</h1>
These days, drones are equipped with some pretty amazing cameras, and the ones that are built for camera work are very steady in the air, using GPS to hold their position steady, and have camera gimbals that compensate for any tilting of the aircraft. As a result, you can get some excellent video of whales. With cameras that good in the air, we realized we could do two important things using machine learning: we could identify individual whales, and we could do health assessments based on body condition.
<h2 id="whale-research-in-alaska">Whale Research in Alaska</h2>
The good folks at <a href="http://www.alaskawhalefoundation.org/">Alaska Whale Foundation</a> shared their database of known humpback whales that they had been studying in the Alaskan waters for years, and Ocean Alliance shared a lot of the video they had collected over the years with drones, and we got to work. In a few short weeks, we built some software prototypes, and then we set off to Alaska with Ocean Alliance, Parley, and the Alaska Whale Foundation, to test the software out.
To get to the whales, we needed to take a plane, like you’d expect. In Alaska, the planes are a little different.
<div class="figure">
<img src="/images/Cessna-Caravan-800x600.jpg" alt="The seaplane" style="width:600px" />
The seaplane
</div>
And of course, we had lodgings, that were also different from what you’d expect on a normal business trip…
<div class="figure">
<img src="/images/ChristianMiller_glacier-seal-snotbot_intel-54-800x408.jpg" alt="The Glacier Seal, the boat we lived on for a week. Photo courtesy of Christian Miller" style="width:600px" />
The Glacier Seal, the boat we lived on for a week. Photo courtesy of Christian Miller
</div>
Plus the rental car…
<div class="figure">
<img src="/images/rental-car-800x450.jpg" alt="Rental car" style="width:600px" />
Rental car
</div>
It was an amazing experience on many levels.
<div class="figure">
<img src="/images/humpback-whale-600x666.png" alt="Image obtained under NMFS #18686" />
Image obtained under NMFS #18686
</div>

<h2 id="identification">Identification</h2>
Identifying individual whales is important for many reasons. In the case of humpback whales in Alaska, there are two populations: one that migrates to Hawaii in the winter, and one that migrates to Mexico in the winter. The Mexico population is much more endangered, so NOAA asks that researchers avoid interacting with the Mexico population if possible. But how is a researcher to know the whale in front of them is from Mexico or from Hawaii? Only if he or she can identify the individual.
We built a program that could compare a snapshot of a humpback whale’s tail flukes against a database of known whales, using the shape of the ragged trailing edge and the amount of grey or white spotting on the tail, and we were able to identify several whales in Alaska while the drone was still in the air over them, which opens up lots of possibilities. Before this, researchers would usually identify the whales days or months after the fact, if at all.
<h3 id="a-discovery">A discovery</h3>
While we were out on the water, listening to a hydrophone, we heard a whale feeding call. It was very distinctive, and Dr. Fred Sharpe from Alaska Whale <img src="/images/Fred-Sharpe-SE-Alaska-200x267.jpg" title="Dr. Fred Sharpe listening to a hydrophone" style="float:right;padding-left:1em;" /> Foundation said he thought it sounded familiar. He checked his archival database on his laptop, and found a similar call from 20 years earlier, from a whale
named Trumpeter. A few minutes later, a whale surfaced, and then we got a picture of its flukes when it dove. We ran it through our software, and we got a match! Sure enough, it was Trumpeter! So we learned something nobody else has known for sure - humpback whale feeding calls remain stable in adults over decades. It wasn’t a question we set out to answer, but it fell into our laps, purely because we were able to identify the whale swimming in front of us.
<div class="figure">
<img src="/images/trumpeter-600x375.png" title="Matching Trumpeter's tail flukes with our software" alt="Live flukes under NMFS #18636, Database under NMFS #14599" style="float:right;padding-left:1em;" />
“Live” flukes under NMFS #18636, “Database” under NMFS #14599
</div>
<h2 id="health-assessment">Health Assessment</h2>
There are lots of things we don’t know about whales. They’re hard to study because they go underwater and they move around a lot. So even a little bit of knowledge about how they’re doing is hard to come by. We built a program that can analyze video data from a drone camera to evaluate how healthy a humpback whale is based on its length and width. Just like other animals, a really thin whale is probably a sick whale, so this simple observational technique alone is already a pretty important indicator of health.
The hard part of this, for a computer, is actually identifying the whale in a video frame. We trained a specialized convolutional neural network to identify whales in difficult conditions like the waters of Alaska, where the color of the whale isn’t much different from the color of the water, and the water tends to be murky.
<div class="figure">
<img src="/images/whale-bci-600x375.png" title="Estimating the health of a whale with our software" style="float:right;padding-left:1em;" />
</div>
Once the whale pixels are identified, we calculate the area of the whale that’s used to store energy (basically, everything but the head and the flukes), that’s identified in pink in the picture above.
The technique worked pretty well, though it requires considerable input from an expert user in its present form (e.g. what was the altitude of the drone, what is the sex the whale, what day of the season is it, and in the case of a mother, how long has it been since the calf was born). We’ll be working to make it easier to use, since this kind of health assessment is an important first step before any kind of interaction with a whale. In fact, I’m supervising an intern, Kelly Cates of the University of Alaska at Fairbanks, to help extend the tool.
We presented a <a href="/misc/sUAS%20Machine%20Learning%20for%20Whales.pdf">poster</a> about this work at the Society for Marine Mammology in October 2017, and just a couple of months later, Iain and I presented together at the Consumer Electronics Show in Las Vegas, which was quite an experience as well.
<div class="figure">
<img src="/images/CES-snotbot-bryn-iain-600x450.jpg" alt="Photo by Christina Caputo" />
Photo by Christina Caputo
</div>
I just received news that the video that was made about our collaboration was nominated for a Webby award! You can see it <a href="https://www.webbyawards.com/winners/2018/film-video/branded-entertainment/corporate-social-responsibility-branded/below-the-surface/">here</a>.
I also went down to Mexico back in February with Parley and Ocean Alliance to capture images of blue whales and grey whales, to see about expanding the health assessment tool to those species as well. Unfortunately, there were very few of either compared to the numbers that were seen even last year. One hesitates to draw conclusions, but it does underscore the urgency of getting a better understanding of the overall population health of whales, who are so important to our ecosystem, our oceans, and our own survival.
Important Note: Flying drones is an activity that is heavily regulated in many parts of the world, and you should check your local regulations before flying any drone. It is especially important to understand that in the United States, it is illegal to fly a drone anywhere near a whale without a very special permit from the National Oceanic and Atmospheric Administration.]]></description>
<pubDate>Thu, 26 Apr 2018 00:00:00 UT</pubDate>
<guid>http://www.xoltar.org/posts/2018-04-26-whale-expedition/index.html</guid>
<dc:creator>Bryn Keller</dc:creator>
</item>
<item>
<title>Python Class Properties</title>
<link>http://www.xoltar.org/posts/2016-06-27-python-class-properties/index.html</link>
<description><![CDATA[<div class="figure">
<img src="/images/Threeclasses.jpg" style="float:left;padding-right:1em;" />
</div>
Class properties are a feature that people coming to Python from other object-oriented languages expect, and expect to be easy. Unfortunately, it’s not. In many cases, you don’t actually want class properties in Python - after all, you can have first class module-level functions as well, you might very well be happier with one of those.
I sometimes see people claim that you can’t do class properties at all in Python, and that’s not right either. It can be done, and it’s not too bad. Read on!

I’m going to assume here that you already know what class (sometimes called “static”) properties are in languages like Java, and that you’re somewhat familiar with Python metaclasses.
To make this feature work, we have to use a metaclass. In this example, we’ll suppose that we want to be able to access a list of all the instances of our class, as well as reference to the most recently created instance. It’s artificial, but it gives us a reason to have both read-only and read-write properties. We define a metaclass, which is again a class that extends <code>type</code>.
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">
class Extent(type):
@property
def extent(self):
ext = getattr(self, '_extent', None)
if ext is None:
self._extent = []
ext = self._extent
return ext
@property
def last_instance(self):
return getattr(self, '_last_instance', None)
@last_instance.setter
def last_instance(self, value):
self._last_instance = value
</code></pre></div>
Please note that if you want to do something like this for real, you may well need to protect access to these shared class properties with synchronization tools like <a href="https://docs.python.org/3.5/library/threading.html#rlock-objects">RLock and friends</a> to prevent different threads from overwriting each others’ work willy-nilly.
Next we create a class that uses that metaclass. The syntax is different in Python 2.7, so you may need to adjust if you’re working in an older version.
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">class Thing(object, metaclass=Extent):
def __init__(self):
self.__class__.extent.append(self)
self.__class__.last_instance = self
</code></pre></div>
Another note for real code: these references (the <code>extent</code> and the <code>last_instance</code>) will keep your object from being garbage collected, so if you actually want to keep extents for your classes, you should do so using something like <a href="https://docs.python.org/3/library/weakref.html">weakref</a>.
Now we can try out our new class:
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">
>>> t1 = Thing()
>>> t2 = Thing()
>>> Thing.extent
[<__main__.Thing object at 0x101c5d080>, <__main__.Thing object at 0x101c5d2b0>]
>>> Thing.last_instance
<__main__.Thing object at 0x101c5d2b0>
>>> </code></pre></div>
Great, we have what we wanted! There are a couple of things to remember, though:
<ul>
<li>Class properties are inherited!</li>
<li>Class properties are not accessible via instances, only via classes.</li>
</ul>
Let’s see an example that demonstrates both. Suppose we add a new subclass of <code>Thing</code> called <code>SuperThing</code>:
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">>>> class SuperThing(Thing):
... @property
... def extent(self):
... return self.__class__.extent
...
>>> s = SuperThing()</code></pre></div>
See how we created a normal <code>extent</code> property that just reads from the class property? So we can now do this:
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">>>> s.extent
[<__main__.Thing object at 0x101c5d080>, <__main__.Thing object at 0x101c5d2b0>, <__main__.SuperThing object at 0x101c5d2e8>]</code></pre></div>
Whereas if we were to try that with one of the original <code>Thing</code>s, it wouldn’t work:
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">>>> t1.extent
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Thing' object has no attribute 'extent'</code></pre></div>
We can of course still access either one via classes:
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">>>> t1.__class__.extent
[<__main__.Thing object at 0x101c5d080>, <__main__.Thing object at 0x101c5d2b0>, <__main__.SuperThing object at 0x101c5d2e8>]
>>> s.__class__.extent
[<__main__.Thing object at 0x101c5d080>, <__main__.Thing object at 0x101c5d2b0>, <__main__.SuperThing object at 0x101c5d2e8>]
>>> </code></pre></div>
Also note that the extent for each of these classes is the same, which shows that class properties are inherited.
Did you spot the bug in <code>Thing</code>? It only manifests when we have subclasses like <code>SuperThing</code>. We inherited the <code>__init__</code> from <code>Thing</code>, which adds each new instance to the extent, and sets <code>last_instance</code>. In this case, <code>self.__class__.extent</code> was already initialized, on <code>Thing</code>, and so we added our <code>SuperThing</code> to the existing list. For <code>last_instance</code>, however, we assigned directly, rather than first reading and appending, as we did with the list property, and so <code>SuperThing.last_instance</code> will be our <code>s</code>, and <code>Thing.last_instance</code> will be our <code>t2</code>. Tread carefully, it’s easy to make a mistake with this kind of thing!
Hopefully this has been a (relatively) simple example of how to build your own class properties, with or without setters.]]></description>
<pubDate>Mon, 27 Jun 2016 00:00:00 UT</pubDate>
<guid>http://www.xoltar.org/posts/2016-06-27-python-class-properties/index.html</guid>
<dc:creator>Bryn Keller</dc:creator>
</item>
<item>
<title>Mac OS X C++ Development (part 2)</title>
<link>http://www.xoltar.org/posts/2016-05-03-mac-os-x-native-2/index.html</link>
<description><![CDATA[<div class="figure">
<img src="/images/charlie-chaplin.jpg" title="C++ on Mac can make you sad" style="float:right;padding-left:1em;" />
</div>
As I wrote in my <a href="/posts/2016-04-21-mac-os-x-native/">previous post</a>, I recently started working on a Mac. This is a collection of my notes on the problems and surprises I ran into. Maybe it will be useful for you, too.

<h1 id="tools">Tools</h1>
In addition to many of the command line tools you may be familiar with from C++ development on Linux, Mac OS X has specialized tools for working with binaries.
<h2 id="otool">otool</h2>
The <code>otool</code> utility displays information from binary headers. It works with Mach headers (which are the main thing, on Macs) but also works with other “universal” formats as well. There are many options, but the main one that’s been helpful to me so far has been <code>otool -L <lib></code>, which tells you the dependencies of the archive, which are essential debugging information for dynamic linking problems. Another useful one is <code>otool -D <lib></code>, which tells you the install names embedded in a given library.
<h2 id="install_name_tool">install_name_tool</h2>
This tool allows you to change the install names for a binary. Simple, but important.
By now you must be wondering what install names are.
<h1 id="install-names">Install Names</h1>
Let us begin with <code>man dyld</code>. There is no <code>dyld</code> command. Nevertheless, it is the dynamic linker for Mac OS X. The man page tells us many interesting things, perhaps chief among which is how dynamic libraries are located at run-time.
When you link an application to a dynamic library, the path to that library is encoded in the application binary. Well, actually, when you build, the library is interrogated to find out where it is supposed to be installed, and that path gets encoded in the application binary. These paths where things are supposed to be installed are known as install names on Mac OS X. Every .dylib has (at least) one. You can use <code>otool -D</code> to find the install name of a .dylib file.
In most cases, the install name will have an absolute path, like this:
<pre><code>> $ otool -D /usr/lib/libiconv.dylib
/usr/lib/libiconv.dylib:
/usr/lib/libiconv.2.dylib</code></pre>
So if you link with /usr/lib/libiconv.dylib, your application will look for /usr/lib/libiconv.2.dylib when it launches. If the .dylib isn’t where it’s expected, the app crashes.
<h2 id="relative-paths">Relative paths</h2>
Sometimes a library will have a relative path as its install name. When this happens, dyld will search for a library with that name on several search paths. It’s similar to <code>LD_LIBRARY_PATH</code> in some ways. There are environment variables you can set to control where it searches. There are several, and you should consult <code>man dyld</code> if you want the gory details. One thing that is important to know is that one of these variables, <code>DYLD_FALLBACK_LIBRARY_PATH</code>, has the default value <code>$(HOME)/lib:/usr/local/lib:/lib:/usr/lib</code>. Occasionally you may see advice on the web that recommends symbolic linking a library into your <code>$(HOME)/lib</code> without any further explanation. <code>DYLD_FALLBACK_LIBRARY_PATH</code> is why.
<h2 id="rpath">@rpath</h2>
If you want to distribute an application that relies on libraries that are not necessarily in known standard places, it is a good idea to use <code>@rpath</code> as part of your install name. This directs dyld to look in a path that is relative to the application binary (or in the case of a library that is dynamically linked to another library, the path is relative to the calling library), rather than looking in the places it normally would. This would allow you to bundle a set of libraries along with your application, and have them be found regardless of where you install the application. There are several other options, such as <code>@executable_path</code> and <code>@loader_path</code>, but <code>@rpath</code> seems to be the right choice in most cases. Again, <code>man dyld</code> has all the details.
<h1 id="an-example-dionysus">An Example: Dionysus</h1>
I wanted to build <a href="http://www.mrzv.org/software/dionysus/">Dionysus</a>, a library for working with <a href="https://en.wikipedia.org/wiki/Persistent_homology">persistent homology</a>. I wanted to build the Python bindings.
<h2 id="linker-errors">Linker Errors</h2>
After I managed to get through the compile phase, I ran into linker problems, with unresolved symbol errors similar to the ones I mentioned in <a href="/posts/2016-04-21-mac-os-x-native">part 1</a>. However, in this case, changing the compiler didn’t fix the problem. I was getting linking errors trying to link with <a href="http://www.boost.org/">Boost</a>, which I had already reinstalled on my system from source, using gcc:
<pre><code>export HOMEBREW_CC=gcc-5
export HOMEBREW_CXX=g++-5
brew reinstall --build-from-source boost
brew reinstall --build-from-source boost --with-python3</code></pre>
That wasn’t enough though. I still got linker errors. Googling suggested maybe the problem was that Boost.Python (which Dionysus uses for its Python binding) was linked to a different version of Python than the one I was targeting. This turned out to be a red herring, however. As far as I can tell, Boost.Python doesn’t actually link to Python at all (though it has to be compiled with support for Python 3 if you want that, so there’s certainly a connection, just not a linking one).
The problem actually was that Dionysus wanted the C++ 11 version of Boost, and the default Homebrew version doesn’t have that support. For this, you need:
<pre><code>brew reinstall --build-from-source boost --with-c++11
brew reinstall --build-from-source boost-python --with-c++11</code></pre>
Build-time linking errors resolved, I thought I was home free.
<h2 id="dynamic-linking-errors">Dynamic Linking Errors</h2>
At last, I had built the library. I fired up python2.7, imported the library. It worked! Then I created an instance of one of the classes, and got this:
<pre><code>TypeError: __init__() should return None, not 'NoneType'</code></pre>
After a lot of googling, it turned out that the most likely cause for this was that the Dionysus binding had wound up linked to the wrong Python library. I carefully checked the CMakeCache.txt file and eliminated any occurrences of the wrong Python library, or the wrong Boost library. Still no luck.
A look at <code>otool -L</code> output for the library showed something funny:
<pre><code>> $ otool -L lib_dionysus.dylib
lib_dionysus.dylib:
/Users/me/src/Dionysus/build/bindings/python/lib_dionysus.dylib (compatibility version 0.0.0, current version 0.0.0)
libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0)
/usr/local/opt/boost-python/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/local/opt/mpfr/lib/libmpfr.4.dylib (compatibility version 6.0.0, current version 6.3.0)
/usr/local/opt/gmp/lib/libgmp.10.dylib (compatibility version 14.0.0, current version 14.0.0)</code></pre>
Do you notice that all the entries are absolute paths except for the libpython2.7 one? A quick look at the Anaconda libpython2.7.dylib I linked against shows why:
<pre><code>> $ otool -D /Users/me/anaconda2/lib/libpython2.7.dylib
/Users/me/anaconda2/lib/libpython2.7.dylib:
libpython2.7.dylib
</code></pre>
So Anaconda’s version of Python instructed the linker to find libpython2.7.dylib by going through the normal dyld process - checking DYLD_LIBRARY_PATH, then DYLD_FALLBACK_LIBRARY_PATH, and so on. Since /usr/lib is on DYLD_FALLBACK_LIBRARY_PATH by default, and the system libpython2.7.dylib is in that directory, Dionysus was getting linked with that one, not with the one that was already in memory in the current process. This led to the strange error I saw.
I was able to use <code>install_name_tool</code> to change the binary to point to the right libpython2.7:
<pre><code>> $ install_name_tool lib_dionysus.dylib -change libpython2.7.dylib /Users/me/anaconda2/lib/libpython2.7.dylib</code></pre>
After this, everything was fine.
<h1 id="further-reading">Further reading</h1>
In addition to <code>man dyld</code>, these are useful:
<ul>
<li><a href="https://www.mikeash.com/pyblog/friday-qa-2009-11-06-linking-and-install-names.html">Linking and Install Names</a></li>
<li><a href="https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html">Runpath Dependent Libraries</a></li>
<li><a href="http://stackoverflow.com/questions/18139710/using-c11-in-macos-x-and-compiled-boost-libraries-conundrum">Using C++11 in Mac OS X with compiled Boost libraries</a></li>
<li><a href="http://stackoverflow.com/questions/20437769/boost-python-init-should-return-none-not-nonetype">Boost Python init should return None not NoneType</a></li>
<li><a href="http://stackoverflow.com/questions/26439319/relative-paths-in-otool-output">Relative paths in otool output</a></li>
</ul>]]></description>
<pubDate>Tue, 03 May 2016 00:00:00 UT</pubDate>
<guid>http://www.xoltar.org/posts/2016-05-03-mac-os-x-native-2/index.html</guid>
<dc:creator>Bryn Keller</dc:creator>
</item>
<item>
<title>Mac OS X C++ Development</title>
<link>http://www.xoltar.org/posts/2016-04-21-mac-os-x-native/index.html</link>
<description><![CDATA[<div class="figure">
<img src="/images/black-and-white-person-feeling-smiling.jpg" title="C++ on Mac can be crazy-making" style="float:right;padding-left:1em;" />
</div>
I recently switched to a MacBook Pro. I have customers that use Linux and Mac, and I wanted to work in a similar environment. Also recently (a few months before the MacBook) I started working with C++ again after a long hiatus.
I had thought that the Mac, being a Unix, would be relatively close to Linux, and that things I was building for Linux would be much more likely to work there than on Windows. That might still be true, but it turns out that there are several things on Mac that are not obvious, and seriously complicate native code development compared with Linux. These are my notes on those differences and how to deal with them. Hopefully, it may be useful for other migrants to Mac as well.

<h1 id="xcode">Xcode</h1>
Apple includes something called Xcode. This is apparently a combination of a platform SDK, and an IDE with a user interface similar to iTunes. You have to have it, but you don’t have to use the IDE part. It must be installed from the App Store. Don’t fight it, just install it and move on.
<h2 id="command-line-tools">Command line tools</h2>
You definitely want the Xcode command line tools. Run:
<pre><code>xcode-select --install</code></pre>
to install them. This will give you git as well.
<h1 id="brew">Brew</h1>
There are actually two package managers for Mac OS X, <a href="https://www.macports.org/">MacPorts</a> and <a href="http://brew.sh/">Homebrew</a>, and as a developer you’ll definitely need one of them. I use brew, because other people I know recommended it, and it’s been nice so far. You need it to install libraries and tools that don’t come with the Mac. Most notably gcc, cmake, and so on.
<h1 id="clang-and-gcc">Clang and gcc</h1>
Apple ships the <a href="http://clang.llvm.org/">clang</a> compiler with Mac OS X, so this is the considered the standard compiler for the platform. This means that some libraries (notably <a href="http://clang.llvm.org/">Qt</a>) only support building with clang.
Some C/C++ projects assume (incorrectly) that everybody builds with gcc. For this reason (I guess), Apple did a really odd thing: they ship a gcc executable, which is actually clang in disguise:
<pre><code>> $ gcc
clang: error: no input files</code></pre>
This (I guess) works sometimes, since many flags work the same in both compilers. However, it is deeply confusing and causes problems as well. For example, gcc supports <a href="http://openmp.org/wp/">OpenMP</a>, a powerful parallel computing tool, and crucial for the work I’m doing. Recent versions of clang support it as well, but Apple’s fork of clang that ships with Macs does not. So to use OpenMP, I have to have the real gcc. This will cause other problems down the road, we’ll get to them in a bit.
You’ll want to install gcc with brew:
<pre><code>brew install gcc gdb</code></pre>
Since clang is already masquerading as gcc, the Homebrew folks came up with a workaround - the gcc package installs executables called gcc-5 and g++-5 instead of gcc and g++. I added the following in my profile to encourage build systems to use these compilers instead of clang.
<pre><code>export HOMEBREW_CC=gcc-5
export HOMEBREW_CXX=g++-5
export CC=gcc-5
export CXX=g++-5
</code></pre>
Note the Homebrew-specific ones. Homebrew generally installs binary, precompiled packages rather than compiling on your machine, but you can pass <code>--build-from-source</code> to it to make it recompile. If you do that, it will honor the <code>HOMEBREW_CC</code> and <code>HOMEBREW_CXX</code> environment variables and use those to do the build.
I also aliased cmake to ensure that cmake uses gcc-5 and g++-5 by default as well:
<pre><code>alias cmake=/usr/local/bin/cmake -DCMAKE_C_COMPILER=$CC -DCMAKE_CXX_COMPILER=$CXX</code></pre>
<h2 id="compatibility">Compatibility</h2>
C++, unlike C, doesn’t specify a binary interface standard. This means that libraries that are compiled with different C++ compilers can have problems interoperating. So there’s that to consider when you use things that were compile with clang (such as anything you download using brew without recompiling) together with things you’ve been building with g++-5.
The most pressing example of this is related to the C++ standard library. There are, on Mac (and elsewhere too I suppose), at least two implementations: libstdc++, and libc++. By default, clang uses libc++ and gcc-5 uses libstdc++. In practice, this means that if you install a C++ based library with brew, you will be able to compile against it with g++-5, but when you get to the link stage, it will fail with lots of missing symbols. If this happens,
<pre><code>brew reinstall --build-from-source <package></code></pre>
can often fix the problem. However, there are brew packages (e.g. pkg-config) that will fail to compile under g++-5, so there can be cases where this doesn’t work. One example: I was trying to build <a href="https://github.com/dmlc/mxnet">mxnet</a> directly using the brew package for <a href="http://opencv.org/">OpenCV</a> support, and it failed with the aforementioned link errors. I tried to reinstall opencv with <code>--build-from-source</code> with brew, and it started recompiling all of opencv’s (many) dependencies, including pkg-config, which for some reason fails to compile. So in the end I had to pull opencv as well and build it manually, after which mxnet built fine too.
<h2 id="next-time">Next time</h2>
These were some important things to be aware of when starting to develop in C++ on Macs. In the <a href="/posts/2016-05-03-mac-os-x-native-2">next installment</a>, we’ll talk about dynamic libraries, install names, and other such loveliness.]]></description>
<pubDate>Thu, 21 Apr 2016 00:00:00 UT</pubDate>
<guid>http://www.xoltar.org/posts/2016-04-21-mac-os-x-native/index.html</guid>
<dc:creator>Bryn Keller</dc:creator>
</item>
<item>
<title>Higher Order Type Theory Resources</title>
<link>http://www.xoltar.org/posts/2015-07-15-hott-resources/index.html</link>
<description><![CDATA[Recently there’s been an outpouring of useful resources folks have found for learning Homotopy Type Theory (aka HoTT). These have come mainly from the HoTT Cafe Google Group.
The main resource is of course, the <a href="http://homotopytypetheory.org/book/">book</a>, but these videos and other resources seem useful enough that I want a consolidated list of them for myself. So I thought maybe you might, too.

Group threads that provided these links:
<ul>
<li><a href="https://groups.google.com/forum/#!topic/hott-cafe/VHzQWyYbXt0">Long way to the HoTT book</a></li>
<li><a href="https://groups.google.com/forum/#!topic/hott-cafe/7zN3y0oSKq4">HoTT without homotopy?</a></li>
<li><a href="https://groups.google.com/forum/#!topic/hott-cafe/kZMjhzFnzHE">Robert Harper’s HoTT course at CMU</a></li>
</ul>
Note that I’m likely to update this page in the future with other resources as I find them, and those links may or may not come from the Groups threads just mentioned.
Useful resources, in no particular order:
<ul>
<li><a href="http://www.mathtube.org/lecture/video/univalence-new-principle-logic">Univalence as a new Principle of Logic</a> - recommended by one writer as perhaps the first video to watch</li>
<li><a href="http://homepages.inf.ed.ac.uk/wadler/papers/propositions-as-types/propositions-as-types.pdf">Wadler’s Propositions as Types paper</a> - recommended by one writer as a good pairing with chapter 1 of the hott book.</li>
<li><a href="http://www.cs.cmu.edu/~rwh/courses/hott/">Robert Harper’s 15-819 Homotopy Type Theory</a>(<a href="http://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx?folderID=07756bb0-b872-4a4a-95b1-b77ad206dab3">videos</a></li>
<li><a href="http://www.amazon.com/Category-Theory-Oxford-Logic-Guides/dp/0199237182">Awodey’s Category Theory book</a></li>
<li><a href="https://www.youtube.com/watch?v=BF6kHD1DAeU">Awodey’s Category Theory lecture videos</a> (URL is to first video, it’s a series)</li>
<li><a href="http://www.bristol.ac.uk/arts/research/projects/homotopy-type-theory/">Bristol HoTT project</a></li>
</ul>
Related:
<ul>
<li><a href="https://github.com/simhu/cubical">Cubical models in Haskell</a></li>
<li><a href="https://github.com/mortberg/cubicaltt">Cubical type theory implementation</a></li>
<li><a href="http://siddhartha-gadgil.github.io/LogicTypesSpaces/">Possible pre-hott intro material</a> - course material, author mentions it may be hard to follow as a standalone resource</li>
<li><a href="http://ncatlab.org/nlab/show/structural+set+theory">Structural set theory</a></li>
</ul>]]></description>
<pubDate>Wed, 15 Jul 2015 00:00:00 UT</pubDate>
<guid>http://www.xoltar.org/posts/2015-07-15-hott-resources/index.html</guid>
<dc:creator>Bryn Keller</dc:creator>
</item>
<item>
<title>Working with Hakyll</title>
<link>http://www.xoltar.org/posts/2015-07-09-hakyll/index.html</link>
<description><![CDATA[<div class="figure">
<img src="/images/haskell-logo.png" />
</div>
WordPress was nice for some time. There were lots of nice themes to choose from, installation was easy, things mostly worked. However, the need to keep updating it with security patches, and the need to keep sifting through comment spam, became tiresome.
I always wish I had time to work on something in Haskell, so I decided to put the need for a new blog platform together with that: I’ve switched to <a href="http://jaspervdj.be/hakyll/">Hakyll</a>, at least for now.

This means I generate the site statically, as HTML pages. This means my site requires very little compute power to run, and it’s even easier for me to administer than the WP site was. It also means I don’t have support for comments, which is sad but saves me lots of time sifting through spam. My contact information is on the <a href="/contact">contact page</a>, please reach out directly if you have comments. Maybe I’ll add some third-party discussion site pointers at a future date. Let me know if you have opinions about that.
Creating the site was fairly simple and took about a day, and took the following steps:
<ul>
<li>Download and install <a href="https://github.com/commercialhaskell/stack">stack</a> (not cabal). This is the build tool we’ll use to fetch and install hakyll and its dependencies. Stack is now the preferred build tool for Haskell. They have binaries available for lots of platforms.</li>
<li>Follow the <a href="http://jaspervdj.be/hakyll/tutorials/01-installation.html">hakyll installation tutorial</a>, substituting “stack” for “cabal”.</li>
</ul>
Now you have a (very) bare bones site. I then:
<ul>
<li>Added <a href="http://getbootstrap.com/examples/blog/">Bootstrap</a>. I basically copied in the HTML from the template and modified it as needed to make it work with Hakyll.</li>
<li>Added RSS and Atom feeds using the <a href="http://jaspervdj.be/hakyll/tutorials/05-snapshots-feeds.html">snapshots tutorial</a></li>
<li>Changed the landing page to show teasers rather than full articles or just links. The <a href="http://jaspervdj.be/hakyll/tutorials/using-teasers-in-hakyll.html">teaser tutorial</a> was very helpful for that. Definitely read this after getting the RSS feeds working, it will make more sense.</li>
<li>Other minor things like adding favicons</li>
</ul>
All in all, quite a bit simpler than I expected. The fact that it was so easy to use stack instead of cabal was a nice surprise along the way. The tutorials for Hakyll are really helpful, though there are some details missing, so be patient.
It turns out not to need as much actual Haskell programming time as I expected, but maybe when I need to make something fancier I’ll get to work on an actual Haskell project.]]></description>
<pubDate>Thu, 09 Jul 2015 00:00:00 UT</pubDate>
<guid>http://www.xoltar.org/posts/2015-07-09-hakyll/index.html</guid>
<dc:creator>Bryn Keller</dc:creator>
</item>
</channel>
</rss>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

Download the "valid RSS" banner.
Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)
Add this HTML to your page (change the image src attribute if necessary):

<a href="http://www.feedvalidator.org/check.cgi?url=http%3A//www.xoltar.org/rss.xml"><img src="valid-rss-rogers.png" alt="[Valid RSS]" title="Validate my RSS feed" /></a>

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//www.xoltar.org/rss.xml

Home · About · News · Docs · Terms