FEED Validator

for Atom and RSS and KML

Congratulations!

This is a valid Atom 1.0 feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

line 5, column 58: Self reference doesn't match document location [help]

 <link href="http://rkj.github.com//atom.xml" rel="self"/>
                                                          ^

line 63, column 0: content should not contain data-lang attribute (21 occurrences) [help]
```
&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;la ...
```

Source: http://feeds.feedburner.com/RkjTechBlog

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>rkj tech blog</title>
<link href="http://rkj.github.com//atom.xml" rel="self"/>
<updated>2017-03-22T23:18:18+00:00</updated>
<id>http://rkj.github.com/</id>
<author>
<name>Roman Kamyk</name>
<email>roman.kamyk@gmail.com</email>
</author>
<entry>
<title>Parsing huge OpenStreetMap's JOSM file using Nokogiri</title>
<link href="http://rkj.github.com//devcamp/josm/openstreetmap/ruby/parser/tutorial/2011/07/05/Parsing-OpenStreetMap-JOSM-data.html"/>
<updated>2011-07-05T00:00:00+00:00</updated>
<id>http://rkj.github.com//devcamp/josm/openstreetmap/ruby/parser/tutorial/2011/07/05/Parsing-OpenStreetMap-JOSM-data</id>
<content type="html"><h1 id="background">Background</h1>
While attending <a href="/event/ssjs/devmeetigs/2011/07/04/DevCamp.html">SSJS Devcamp</a>
I was having fun with <a href="http://www.mongodb.org">MongoDB</a> and
<a href="http://couchdb.apache.org/">CouchDB</a>. To make some useful tests we had first to gather some data. The plan was to import data into MySQL, next migrate it
to NoSQL dbs and then make some tests/comparisons. After a quick brainstorm we
decided to create database with places/POIs. The greatest free geo repository I
know is <a href="http://openstreetmap.org">OpenStreetMap</a> and they are providing all
theirs data for <a href="http://wiki.openstreetmap.org/wiki/Downloading_data">download</a>.
Whole repository is really huge (almost 17GiB of compressed data), so we
settled for using only Poland data from a <a href="http://download.geofabrik.de/osm/europe/">nice
mirror</a>.
<h1 id="data">Data</h1>
Format specification is described on <a href="http://wiki.openstreetmap.org/wiki/JOSM_file_format">OSM
wiki</a>. It is pretty
straightforward XML and contains mostly roads, but also cities, buildings,
shops and ton of other things. This makes even Poland data pretty big (~150MiB
compressed, ~2.5GiB uncompressed XML file). For us just the POI stuff was
important so we had to go through whole file and cherry pick interesting
entries.
<h1 id="parser">Parser</h1>
Because of the size of the XML, using a <a href="http://en.wikipedia.org/wiki/XML#Document_Object_Model_.28DOM.29">DOM
parser</a> on a
laptop would not work because of memory usage. Obvious solution would be to use
SAX parser, but I consider it to be totally backwards<a href="#fn:1" class="footnote">1</a>. I prefer <a href="http://www.xmlpull.org/">Pull
Parsing</a> so I searched for some nice parser,
preferably in Ruby. As it turns out, the excellent
<a href="http://nokogiri.org">Nokogiri</a> have one as
<a href="http://nokogiri.org/Nokogiri/XML/Reader.html">XML::Reader</a>.
<h1 id="code">Code</h1>
The import was broken into two phases. In phase one the interesting data was
taken from the XML and saved as JSON (it could be any other format of course).
In the second phase it was read and inserted into MySQL. It allowed writing
phase two code while phase one was running and was generally more error
resistant (errors in phase two didn’t cause whole import to be rerun).
Phase one code from <a href="https://github.com/rkj/devcamp-ssjs-db/blob/master/osm/parse-osm.rb">GitHub/parse-osm.rb</a>:
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93</pre></td><td class="code"><pre>#!/usr/bin/env ruby
# Small script for scraping POIs from JOSM (http://wiki.openstreetmap.org/wiki/JOSM_file_format).
require 'rubygems'
require 'bundler/setup'
require 'nokogiri'
require 'json'
require 'ap'
class Parser
# Tags in data that we ignore.
IGNORED_TAGS = ["created_by", "source"]
def initialize
# Map storing attribute name mapped on count of nodes containing it.
# It is helpful to see what tags should be taken into account in the first place
# during importing.
@popular_attributes = {}
# number of nodes parsed
@count = 0
# number of total xml nodes went through
@total_count = 0
# number of entries considered useful as POI
@included = 0
end
def parse(input, output)
out = File.new(output, "w")
begin
out.write "[\n"
# Nokogiri reader created.
reader = Nokogiri::XML::Reader(File.new(input))
while reader = parse_node(reader, out)
end
ensure
out.write "{}\n]\n"
out.close
ap @popular_attributes.sort_by { |k, v| v}.reverse
STDERR.puts ""
puts "\n#{@included} / #{@count} / #{@total_count}\t"
end
end
def parse_node(r, out)
# Search for 'node' tags because they contain data (points). Other tags
# are discarder.
(r = r.read; progress) while r &amp;&amp; r.name != 'node'
# Stop processing if end of file
return false unless r
# Create entry to be enriched with 'tag' data
entry = { :lat =&gt; r.attribute("lat"), :lon =&gt; r.attribute("lon") }
# Required fields to create usable POI.
req = ["name"]
while (progress; r = r.read)
# Next node found, so no more tags.
break if r.name == 'node'
# Only 'tag' are interesting.
next unless r.name == 'tag'
# Each tag has form of &lt;tag k="key" v="value" /&gt;
key = r.attribute "k"
unless IGNORED_TAGS.include? key
req.delete key
entry[key] = r.attribute "v"
@popular_attributes[key] ||= 0
@popular_attributes[key] += 1
end
end
# If all required tags were found.
if req.size == 0
@included += 1
out.write(entry.to_json)
out.write(",\n")
end
progress(true)
return r
end
# Progress info
def progress(entry_found = false)
@total_count += 1;
@count += 1 if entry_found
limit = 10000
if @total_count % limit == 0
STDERR.print "."
STDERR.print "\r#{@included} / #{@count} / #{@total_count}\t" if @total_count % (limit * 50) == 0
STDERR.flush
end
end
end
if ARGV.size &lt; 2
puts "Usage: #{$PROGRAM_NAME} osm_file output_json"
exit 1
end
Parser.new.parse ARGV[0], ARGV[1]
</pre></td></tr></tbody></table></code></pre></figure>
Parsing whole file ran for a little over 250 seconds:
<div class="highlighter-rouge"><pre class="highlight"><code>ruby parse-osm.rb poland.osm poi.json 247.94s user 3.07s system 99% cpu 4:13.08 total
</code></pre>
</div>
Just copying the file take more than half of that:
<div class="highlighter-rouge"><pre class="highlight"><code>cp -i poland.osm del.me 0.03s user 3.80s system 2% cpu 2:32.33 total
</code></pre>
</div>
So the overhead is not too big. While running, the script used less than 4MiB
of RAM, so also very acceptable. The result is 101 793 POI candidates[^2] that
will be imported into the DB in the phase two. It will be described in more
details in a following post, so stay tuned!
react appropriately. Is is just wrong. I guess this API was easy to implement,
but the result is a strange to use, and code using it ends up pretty mangled.
[^2]: Extracted from 14 122 097 JOSM ‘node’s and 89 522 705 XML nodes;that gives processing of
more than 362 440 XML tags per second.
<div class="footnotes">
<ol>
<li id="fn:1">
I mean literally backwards. You, as a client, has to maintain state and&nbsp;<a href="#fnref:1" class="reversefootnote">&#8617;</a>
</li>
</ol>
</div>
</content>
</entry>
<entry>
<title>SSJS Devcamp organized by DevMeetings.pl</title>
<link href="http://rkj.github.com//event/ssjs/devmeetigs/2011/07/04/DevCamp.html"/>
<updated>2011-07-04T00:00:00+00:00</updated>
<id>http://rkj.github.com//event/ssjs/devmeetigs/2011/07/04/DevCamp</id>
<content type="html">I have always liked lightweight events, like BarCamp, and have tried to attend
most of locally organized ones. At the beginning of this year I have taken
part in <a href="http://www.jug.poznan.pl/materialy-ze-spotkan/materialy/1-poznanski-code-retreat/">Poznań Code
Retreat</a>
that was really great and intense coding experience, and I would love to take
part in another one.
In June I have received an invitation to a Server Side Javascript R&amp;D Camp
organized by <a href="http://devmeetings.pl">DevMeetings.pl</a>. It was advertised as a
three days long, advanced and practical developers meeting in a nice hotel in
countryside. It didn’t take me long to accept the invitation and I was eagerly
waiting for it.

About twenty developers arrived on Thursday evening. For the first two days we
worked in 4-5 people teams researching different topics: security, performance,
NoSQL databases, integration, tools and practices. I worked on db team and will
describe the results in some following post. On each day’s evening each team
presented theirs results. This introduced a little pressure and made people
work hard, as no one would like to present nothing…
On Sunday, our final day, each team was given the same task: implement simple
twitter clone using some of the technologies we have worked with. We started at
10am and the deadline was dinner at 2pm. I, together with
<a href="http://twitter.com/#!/bartaz">Bartek</a> and
<a href="http://twitter.com/#!/gustaff_weldon">Bernard</a> used RingoJS deployed on
AppEngine, and the results are visible
<a href="http://devcamp.ringojs-twitter.appspot.com/">here</a>. I think it is quite
impressive that now we have technologies and tools that allows us to build such
things in a rather short time span. But to be honest we could use Python
instead of RingoJS and there would be even less work to do
(<a href="http://www.appenginejs.org/">appenginejs</a> is rather incomplete and needed
some patches).
And again, after dinner each team showed theirs app, each was crowd beta-tested
and the results of the camp was discussed. Sadly I had to leave before
the conclusion but I have a feeling that SSJS was not voted as production-ready
Java replacement… But more on that later.
I would like to thank the organizers and all the great attenders, I had a great
time and learned a lot about today’s hot technologies. See you next time!
</content>
</entry>
<entry>
<title>Email notification for multiple Mercurial repositories</title>
<link href="http://rkj.github.com//mercurial/notifications/2010/06/10/MercurialMultipleReposEmail.html"/>
<updated>2010-06-10T00:00:00+00:00</updated>
<id>http://rkj.github.com//mercurial/notifications/2010/06/10/MercurialMultipleReposEmail</id>
<content type="html"><h1 id="mercurial-server">Mercurial-server</h1>
Recently I have discovered the awesome <a href="http://www.lshift.net/mercurial-server.html">mercurial-server</a>. It gives flexible access control, takes care of file permission (which was problematic in our previous setup) and allows to hide physical path of the repo from its URL. It also offers global configuration file, which can include hooks for all of the repositories.
This offers an advantage in setting mail notifications using <a href="http://mercurial.selenic.com/wiki/NotifyExtension">NotifyExtension</a>. In our previous setup we had included all the options and the template in each’s repository hgrc file. It required us to use some setup script for creating repositories and was somewhat clumsy - there were no feel of some global administration.
Using global configuration allow us to easily put the configuration in the global rc file (in <code class="highlighter-rouge">/etc/mercurial-server/remote-hgrc.d/</code> dir) pointing to the template inside <code class="highlighter-rouge">hgadmin</code> repo. There would still be need to put <code class="highlighter-rouge">usersubs</code> or <code class="highlighter-rouge">reposubs</code> sections in each repository’ hgrc to specify who gets notification from which repo but it is now much cleaner and only info about specific repo goes to the hgrc.
<h1 id="global-subscribers-configuration">Global subscribers configuration</h1>
We wanted to get one step further and have some global configuration of subscribers in one place (and preferably in <code class="highlighter-rouge">hgadmin</code> repo), but this required some extra effort. Since I have not found a way to use any variables in <code class="highlighter-rouge">reposubs</code> section (which specifies who gets notified) we decided that all emails will go to local account (<code class="highlighter-rouge">hg@localhost</code> at the server), and will include custom header (X-Repo) with the repository name (it was easily added in the style file).
Then we use <a href="http://www.procmail.org/">procmail</a> to handle routing:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">:0c
* ^X-Repo: \/.*
{
RPATH=/repos/hgadmin/subscriptions/${MATCH}
DEFAULT=`cat /repos/hgadmin/subscriptions/default`
EMAIL=`if [ -f $RPATH ]; then cat $RPATH; else echo "Please create file /hgadmin/subscriptions/${MATCH}!" | mail -s "No subscriptions for repository &gt;&gt;${MATCH}&lt;&lt;" $DEFAULT; echo $DEFAULT ; fi `
:0
! ${EMAIL}
}</code></pre></figure>
If email have the <code class="highlighter-rouge">X-Repo</code> header we search for subscribers in the repository’s file (addresses can be separated by spaces or newlines). If there is no relevant file, a warning email is sent to the default address, so we do not forget about any created repo.
Then procmail forwards the notification email to the subscribers and we are happy. Of course one could have even more advanced setup involving reading groups or privileges from LDAP, etc.
But maybe you know a simpler way (without procmail) to achieve this?
Below you see our configuration files:
<h1 id="etcmercurial-serverremote-hgrcdnotifyrc">/etc/mercurial-server/remote-hgrc.d/notify.rc</h1>
<figure class="highlight"><pre><code class="language-ini" data-lang="ini">[extensions]
hgext.notify =
[hooks]
changegroup.notify = python:hgext.notify.hook
[email]
from = vcs-hg@itiner.pl
[smtp]
host = localhost
[notify]
test = false
strip = 0
style = /repos/hgadmin/styles/notify.style
[reposubs]
# key is glob pattern, value is comma-separated list of subscriber emails
* = hg@localhost</code></pre></figure>
<h1 id="reposhgadminstylesnotifystyle">/repos/hgadmin/styles/notify.style</h1>
<figure class="highlight"><pre><code class="language-ini" data-lang="ini">start_files = 'files: '
file = ' {file}'
end_files = '\n'
start_file_mods = 'files: '
file_mod = ' {file_mod}'
end_file_mods = '\n'
start_file_adds = 'files+: '
file_add = ' {file_add}'
end_file_adds = '\n'
start_file_dels = 'files-: '
file_del = ' {file_del}'
end_file_dels = '\n'
start_file_copies = 'copies: '
file_copy = ' {name} ({source})'
end_file_copies = '\n'
extra = 'extra: {key}={value|stringescape}\n'
changeset="X-Repo: {webroot|basename}\nSubject: [{webroot|basename}] Mercurial changeset {rev} by {author}\n\nrepository: https://vcs.itiner.pl/projects/{webroot|basename}/repository\nchangeset: {rev}\nuser: {author}\ndate: {date|date}\n{file_mods}{file_adds}{file_dels}{file_copies_switch}{extras}\ndescription:\n{desc}\n"</code></pre></figure>
</content>
</entry>
<entry>
<title>Automatically adding specific files before commit to Mercurial repo</title>
<link href="http://rkj.github.com//mercurial/2010/05/26/Mercurial-PreCommitAddFiles.html"/>
<updated>2010-05-26T00:00:00+00:00</updated>
<id>http://rkj.github.com//mercurial/2010/05/26/Mercurial-PreCommitAddFiles</id>
<content type="html">When you create new files, you have to add them manually to the repository using either <code class="highlighter-rouge">hg add</code> or <code class="highlighter-rouge">hg addremove</code> before issuing commit.
If you often create some files you may want not to be bothered by this additional commands. Then you may use <code class="highlighter-rouge">hg commit -A</code> which will add all new files automatically. You can even make an alias in your <code class="highlighter-rouge">.hgrc</code> so every commit will have the option turned on.
But there might be a problem if you have some unnecessary files (like logs) that are not specified in <code class="highlighter-rouge">.hgignore</code>.
If for some reason you cannot/do not want to create ignore rules for them, but you still want to add some files automatically with commit you may try a simple hook in your <code class="highlighter-rouge">.hgrc</code>:
<figure class="highlight"><pre><code class="language-ini" data-lang="ini">[hooks]
pre-commit = hg add . -I 'glob:**.c' -q || true</code></pre></figure>
Where you should of course change <code class="highlighter-rouge">'glob:**.c'</code> to a pattern that will match your files.
And do not confuse <code class="highlighter-rouge">pre-commit</code> with <code class="highlighter-rouge">precommit</code> as they are totally different.
Personally I do not use this, because always before issuing a commit I do:
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">$ hg status
$ hg diff | mate </code></pre></figure>
to see all the changes made and make sure that there are no leftover changes (like some random debugging printfs). But your situation may be different and you can give the hook a try.
</content>
</entry>
<entry>
<title>Mercurial rollback can cause unexpected data loss</title>
<link href="http://rkj.github.com//mercurial/2010/02/18/Mercurial-rollback-dataloss.html"/>
<updated>2010-02-18T00:00:00+00:00</updated>
<id>http://rkj.github.com//mercurial/2010/02/18/Mercurial-rollback-dataloss</id>
<content type="html">One of import differences between Mercurial and git is that in the former it is very hard to change the history of a repository. It is by design (although it can be ,,bypassed’’ with usage of plugins like MQ) and there are good reasons for it. This gives this warm, fuzzy feeling of security and makes harder to screw up really hard. So I consider it to be a real user-friendly aspect of Mercurial.
But recently I was bitten by rollback command which undo last transaction (<code class="highlighter-rouge">commit</code>, <code class="highlighter-rouge">pull</code>, and some other commands). I use it more often than I would like - mostly for undoing commit. It helps when I have forgot to add a file, or added some garbage, etc. It is suitable only before pushing changes somewhere, because otherwise they will back. But in my case, many simple mistakes are spotted right away, just after <code class="highlighter-rouge">commit</code>, and maybe <code class="highlighter-rouge">status</code>. They may be fixed without rollback - with a second commit, but it would unnecessary clobber history and make look silly. So if I make an erroneous commit I rollback, fix it and commit again. This procedure has worked fine many, many times. Until recently.
Let see some contrived example. First we create a repo:
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">$ mkdir example
$ cd example
$ hg init
$ echo a &gt; a
$ hg ci -Am 'a'</code></pre></figure>
Next we start some development in a branch:
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">$ hg branch b
$ echo b &gt; b
$ hg ci -Am 'b'</code></pre></figure>
And next we make mistaken commit
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">$ echo d &gt; c
$ hg ci -Am 'c'</code></pre></figure>
Normally if we spot error right away we can issue:
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">$ hg rollback
$ cat c
d # ok, the file is there, we can fix it
$ hg ci -Am 'c'</code></pre></figure>
But if we change a branch and then make a rollback we are screwed:
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">$ hg up default
$ hg rollback
$ cat c
cat: c: No such file or directory # hmmm
$ hg up b # let's go to our original branch
$ cat c
cat: c: No such file or directory # still nothing :/</code></pre></figure>
Since the commit is undone there is no <code class="highlighter-rouge">c</code> in branch <code class="highlighter-rouge">b</code>, so <code class="highlighter-rouge">update</code> does not help. Fortunately for me, I had editor windows still opened, where I could use undo - to redo my changes. This time I have not lost anything. But there were some scary moments ;-).
So I think the lesson is that if you are to make a rollback then be sure that you have not made any updates to the repository, because rollback will not even warn you.
</content>
</entry>
<entry>
<title>Updated Rakefile for dealing with mercurial subrepos</title>
<link href="http://rkj.github.com//mercurial/rake/subrepositories/2010/02/13/Mercurial-subrepos-update.html"/>
<updated>2010-02-13T00:00:00+00:00</updated>
<id>http://rkj.github.com//mercurial/rake/subrepositories/2010/02/13/Mercurial-subrepos-update</id>
<content type="html">Previously I have described my solution for subrepos in Mercurial. It was working nicely but, a little painfully, because you had to add every single Mercurial command you like to use as Rake task. After little googleing and scratching head I have come with a much nicer and general solution:
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby">desc "Invokes any hg command on every repo."
task "hg:command"
rule /^hg:/ do |t| # rule selecting task starting with hg:
on_repos("hg #{t.name.sub(/^hg:/, "")}") # 'hg:' is in task name so we have to erase it
end # and that's all folks :D</code></pre></figure>
It is shorter and any command may be used, with any parameters, like:
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby">rake hg:status
rake hg:"revert -a"</code></pre></figure>
and so on. Enjoy!
</content>
</entry>
<entry>
<title>Shifting bytes left in different languages</title>
<link href="http://rkj.github.com//java/ruby/python/c/programming/2010/01/19/Shift-left.html"/>
<updated>2010-01-19T00:00:00+00:00</updated>
<id>http://rkj.github.com//java/ruby/python/c/programming/2010/01/19/Shift-left</id>
<content type="html">Recently I must have had a really bad day, because I wrote a very stupid line of code in java:
<figure class="highlight"><pre><code class="language-java" data-lang="java">int x = 2 &lt;&lt; (s - 1);</code></pre></figure>
when I wanted something a little simpler:
<figure class="highlight"><pre><code class="language-java" data-lang="java">int x = 1 &lt;&lt; s;</code></pre></figure>
It gave unexpected (for me) behavior for shifting negative number of bits. I was having hope that shifting -1 place left would be equal to shifting 1 place right. But I was wrong, and the result was always zero, no matter what number I would try to shift.
After a little search I have found: <a href="http://java.sun.com/docs/books/tutorial/java/nutsandbolts/op3.html" title="Bitwise and Bit Shift Operators">Bitwise and Bit Shift Operators</a> but is says nothing about this case.
So I tried the same code in different languages to check how they behave. I started with my favorite – Ruby:
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby">def shift(no, s)
[no &lt;&lt; (s - 1), (no &lt;&lt; s) / 2]
end
shift(2, 8) # =&gt; [256, 256]
shift(2, 1) # =&gt; [2, 2]
shift(2, 0) # =&gt; [1, 1]
shift(256, -4) # =&gt; [8, 8]</code></pre></figure>
Here the results were as I would like them to be.
So what does our good, old friend C has to say?
<figure class="highlight"><pre><code class="language-c" data-lang="c">#include &lt;stdio.h&gt;
void shift(int no, int s) {
int one = no &lt;&lt; (s - 1);
int two = (no &lt;&lt; s) / 2;
printf("Shift(%d, %d): %d, %d\n", no, s, one, two);
}
int main() {
shift(2, 8); // 256, 256
 shift(2, 1); // 2, 2
 shift(2, 0); // 0, 1
 shift(256, -4); // 0, 0
 return 0;
}</code></pre></figure>
It gives the same results as Java code. So I have to assume that it is described in some specification, or maybe is processor specific as shifting is simple assembler instruction.
Last language I have tried is Ruby companion, Python, which behave more elegantly than C or Java:
<figure class="highlight"><pre><code class="language-python" data-lang="python">def shift(no) :
return (2 &lt;&lt; (no - 1), (2 &lt;&lt; no) / 2)
print shift(8) # (256, 256)
print shift(1) # (2, 2)
print shift(0) # ValueError: negative shift count</code></pre></figure>
It actually checked for nonsensical value and raised an exception not giving me silently wrong (from my perspective) value.
Sadly Python does not behave similarly for other basic mistakes, as:
<figure class="highlight"><pre><code class="language-python" data-lang="python">&gt;&gt;&gt; 2 &lt; "1"
True</code></pre></figure>
</content>
</entry>
<entry>
<title>Rakefile for dealing with mercurial subrepos</title>
<link href="http://rkj.github.com//mercurial/rake/subrepositories/2010/01/16/Mercurial-subrepos.html"/>
<updated>2010-01-16T00:00:00+00:00</updated>
<id>http://rkj.github.com//mercurial/rake/subrepositories/2010/01/16/Mercurial-subrepos</id>
<content type="html">Subrepositories are quite an important things if you have any bigger project, where you can extract some libraries, modules, parts or any other such things. We have chosen Mercurial for version control, because it distributed (svn suxx ass), easy to use, and have good cross-platform support. Until version 1.3 it did not have any support for subrepositories, now it has one.
It is far from perfect, but still much better than nothing. And I am sure it will get better, or eventually great.
But for the time I have made quite simple Rakefile to help with subrepos. It works fine also with Mercurial versions less than 1.3. To use it you must have Ruby and gems: rake, term-ansicolor installed. Script assumes that you have subdirectories which are Mercurial repositiories and issues commands on each of them.
<code class="highlighter-rouge">rake -T</code> will give you list of available tasks:
<ul>
<li>hg - generic task. You can invoke it like: <code class="highlighter-rouge">CMD=status rake hg</code> and it will invoke CMD on every repository. In this case it will display status.</li>
<li>pull, push, status - does what the name says.</li>
<li>update - does pull -u.</li>
<li>rebase - does pull –rebase. You must have <code class="highlighter-rouge">hgext.rebase=</code> enabled for it to work.</li>
<li>tag - invoked <code class="highlighter-rouge">rake tag name=0.1</code> will tag each repository with tag 0.1.</li>
<li>upto - invoked <code class="highlighter-rouge">rake upto rev=0.2</code> will do <code class="highlighter-rouge">hg up -r 0.2</code>. It makes sense with branches and tags, not so much with revisions, because each subrepo have its own history ;-).</li>
<li>commit - if you have main repository, that contains all the others (as when you use Mercurial subrepo support) it will read commit messages from all the subrepos since last commit to the main repo and copy them into clipboard (works on a Mac only, I think).</li>
</ul>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90</pre></td><td class="code"><pre>begin
require 'term/ansicolor'
class String
include Term::ANSIColor
end
rescue LoadError =&gt; e
STDERR.puts "Please install gem term-ansicolor to use this rakefile."
exit(1)
end
@repos = Dir["**/.hg"].map { |f| f.sub(/.hg$/, "") }
def on_repos(cmd)
@repos.each do |project|
begin
puts "= #{project.red} (#{cmd.yellow})"
system "cd #{project}; #{cmd}"
rescue =&gt; e
puts "Failed because: #{e}".red
end
end
end
desc "Invokes any hg command on every repo. Provide CMD=command env variable."
task :hg do
if ENV['CMD'].nil?
puts "Missing rev parameter"
else
on_repos("hg #{ENV['CMD']}")
end
end
desc "Rebase"
task :rebase do
on_repos("hg pull --rebase")
end
desc "Pull"
task :pull do
on_repos("hg pull")
end
desc "Pull -u"
task :update do
on_repos("hg pull -u")
end
desc "Status"
task :status do
on_repos("hg status")
end
desc "Push"
task :push do
on_repos("hg push")
end
desc "Tag repos. Provide name=tag_name param"
task :tag do |t, args|
if args.name.nil?
puts "Missing name parameter"
else
on_repos("hg tag #{args.name}")
end
end
desc "Update all repos to some revision. Provide rev=revision parameter."
task :upto do |t, args|
if args.rev.nil?
puts "Missing rev parameter"
else
on_repos("hg update -C #{args.name}")
end
end
desc "Commits main repository and adds commits messages from every subrepo."
task :commit do
require 'time'
last_date = Time.parse(%x{hg tip --template '{date|isodate}'}) + 60
date = last_date.strftime("%Y-%m-%d %H:%M")
out = @repos.reject { |r| File.basename(r) == "itiner" }.map do |r|
log = %x{cd #{r} &amp;&amp; hg log -d '&gt;#{date}' --template="== {desc}\n"}.strip
log.empty? ? nil : "= #{File.basename r}\n#{log}"
end
msg = out.compact.join("\n\n").strip
# mac only?
IO.popen('pbcopy', 'w') do |pipe|
pipe.print(msg)
end
system "hg commit &amp;&amp; hg push"
end
</pre></td></tr></tbody></table></code></pre></figure>
</content>
</entry>
<entry>
<title>WYSIWYG in LaTeX with Rakefile</title>
<link href="http://rkj.github.com//latex/rake/2010/01/15/LaTeX-with-Rakefile.html"/>
<updated>2010-01-15T00:00:00+00:00</updated>
<id>http://rkj.github.com//latex/rake/2010/01/15/LaTeX-with-Rakefile</id>
<content type="html">First tools to work with text I have learned where WYSIWYG and everywhere I looked they were praised. I really did not know the alternative and could not think a reason for it to exist in time of fast computers.
But when I was exposed to LaTex I have instantly fell in love. Compared to Word it produces astonishingly beautiful output, takes whole of issues away (numbering, chapters, pages with one sentence on it, positioning pictures and many, many more) and lets you focus on content (and structure) not on the final look (just like WriteBoard, I am using right now). But it has some issues. Actually my relationship with LaTeX is more like love-hate one. Sooner or later you will want to do some non-trivial thing to do. And you will either have to dig really deeply or find someone who did it before - and neither one may be easy, even on the shoulders of google.
But knowing many its faults I still do know imagine writing a thesis in some word processor instead of LaTeX - that would be just plain stupid…
I would like to share with you a simple script that makes life with LaTeX a little nicer. It is a Rakefile, so you need to have Ruby and Rake gem installed. It contains some simple tasks:
<ul>
<li>compile - creates PDF document from the source files. It makes a couple of runs too much, but just to be sure that everything is on the right place, with bibliography, table of content, etc.</li>
<li>clean - deletes all the messy and unneeded files if you would like to zip folder and send it somewhere.</li>
<li>view - tries to display PDF in some reader available on your system.</li>
<li>run - the most interesting part. I have stolen a concept from autotest. It checks the source files every second and if there are any changes it runs compile. If you have a sane PDF reader (like Sumatra on Windows, or Skim on Mac - not Adobe Reader) then it will refresh pdf as soon as it is ready. So you can have this rake task running in the background, and every time you hit save you will see changes in the pdf.</li>
</ul>
I consider this to be the best of both worlds. You can write freely for most time, but when you are finishing and want to take care of the final look you can almost instantly see changes in PDF and tweek quirks until you are satisfied.
There are other tools to acomplish similar results. One of them is LyX, but there is a problem with retrieving source tex files from it. And you will need them at some point, to do some tweeking, or something that is not available in LyX, but can be done in LaTeX and it will be painful. So do not even start - its lame. There are also some editors with similar capabilities, but I have my favorite editor, you may have yours - and the scripts works with them all.
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56</pre></td><td class="code"><pre>TEX_FILES = FileList["*.tex", "figures/*", "*.bib", "*.cls"]
MAIN = "thesis" # name of main file without .tex suffix
MAIN_TEX = "#{MAIN}.tex"
OUT_PDF = "#{MAIN}.pdf"
file OUT_PDF =&gt; TEX_FILES do
%x{pdflatex -interaction=batchmode #{MAIN_TEX} &gt;/dev/null 2&gt;&amp;1}
%x{makeindex #{MAIN} &gt;/dev/null 2&gt;&amp;1}
%x{bibtex #{MAIN} &gt;/dev/null 2&gt;&amp;1}
%x{pdflatex -interaction=batchmode #{MAIN_TEX} &gt;/dev/null 2&gt;&amp;1}
%x{bibtex #{MAIN}.gls &gt;/dev/null 2&gt;&amp;1}
%x{makeindex #{MAIN} &gt;/dev/null 2&gt;&amp;1}
%x{pdflatex -interaction=batchmode #{MAIN_TEX} &gt;/dev/null 2&gt;&amp;1}
%x{makeindex #{MAIN} &gt;/dev/null 2&gt;&amp;1}
%x{bibtex #{MAIN} &gt;/dev/null 2&gt;&amp;1}
%x{bibtex #{MAIN}.gls &gt;/dev/null 2&gt;&amp;1}
%x{pdflatex -interaction=batchmode #{MAIN_TEX} &gt;/dev/null 2&gt;&amp;1}
sh "pdflatex -interaction=batchmode #{MAIN_TEX} &gt;/dev/null"
cp "#{MAIN}.pdf"
end
desc "Removes unnecessery files"
task :clean do
rm FileList["*.aux","*.bak","*.log","*.blg", "*.bbl", "*.toc", "*.out", "*.idx", "*.ilg", "*.ind", "#{THESIS}.pdf", "*.dep", "*.glo", "*.gls"]
end
task :default =&gt; [:run]
"Open text editor to edit files"
task :edit do
["$EDITOR", "mate", "vim", "emacs"].find do |editor|
system "#{editor} *tex *bib Rakefile"
end or puts "Unable to find text editor."
end
task :run do
t = Rake::Task[OUT_PDF]
loop do
begin
t.reenable
t.invoke
rescue =&gt; e
puts "Error: #{e}, sleeping for 15 seconds."
Kernel.sleep 15
end
Kernel.sleep 1
end
end
desc "Shows Thesis"
task :view =&gt; [OUT_PDF] do
["open", "okular", "kpdf", "acroread"].find do |viewer|
system "#{viewer} #{OUT_PDF}"
end or
puts "Unable to find any pdf viewer."
end
</pre></td></tr></tbody></table></code></pre></figure>
</content>
</entry>
</feed>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

Download the "valid Atom 1.0" banner.
Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)
Add this HTML to your page (change the image src attribute if necessary):

<a href="http://www.feedvalidator.org/check.cgi?url=http%3A//feeds.feedburner.com/RkjTechBlog"><img src="valid-atom.png" alt="[Valid Atom 1.0]" title="Validate my Atom 1.0 feed" /></a>

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//feeds.feedburner.com/RkjTechBlog

Home · About · News · Docs · Terms