Congratulations!

[Valid Atom 1.0] This is a valid Atom 1.0 feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

Source: http://stillwater-cse.blogspot.com/feeds/posts/default

  1. <?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?><feed xmlns='http://www.w3.org/2005/Atom' xmlns:openSearch='http://a9.com/-/spec/opensearchrss/1.0/' xmlns:blogger='http://schemas.google.com/blogger/2008' xmlns:georss='http://www.georss.org/georss' xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr='http://purl.org/syndication/thread/1.0'><id>tag:blogger.com,1999:blog-715725806728995718</id><updated>2023-11-15T22:46:10.009-08:00</updated><category term="&quot;cloud computing&quot;"/><category term="SaaS"/><category term="PaaS"/><category term="&quot;Web Services&quot;"/><category term="&quot;computational science&quot;"/><category term="BI"/><category term="cse"/><category term="productivity"/><category term="silicon innovation"/><category term="venture capital"/><category term="&quot;Federated Clouds&quot;"/><category term="&quot;Grid Computing&quot;"/><category term="&quot;IT as a Business&quot;"/><category term="&quot;SaaS Business development&quot;"/><category term="&quot;Saas Investment Scorecard&quot;"/><category term="&quot;adoption-led&quot;"/><category term="&quot;business intelligence&quot;"/><category term="&quot;business model&quot;"/><category term="&quot;data mining&quot;"/><category term="&quot;open source adoption&quot;"/><category term="&quot;provisioning stacks&quot;"/><category term="&quot;release engineering&quot;"/><category term="&quot;saas aggregation&quot;"/><category term="&quot;saas investment&quot;"/><category term="&quot;saas perceptions&quot;"/><category term="&quot;user perspective&quot;"/><category term="&quot;web server&quot;"/><category term="&quot;world wide web&quot;"/><category term="AWS"/><category term="Business Intelligence"/><category term="HPC"/><category term="HaaS"/><category term="IT"/><category term="IaaB"/><category term="JeOS"/><category term="OSFAGPOS"/><category term="Operations Research"/><category term="Predictive Analytics"/><category term="WWW"/><category term="aggregation"/><category term="amazon"/><category term="cloud computing standards"/><category term="codecs"/><category term="competitiveness"/><category term="cost"/><category term="grid"/><category term="investment"/><category term="mashups"/><category term="microsoft"/><category term="mid-market"/><category term="missing middle"/><category term="monetization"/><category term="optimization"/><category term="outsourcing"/><category term="scorecard"/><category term="start-ups"/><category term="supercomputing"/><category term="utility computing"/><title type='text'>High-productivity Cloud computing</title><subtitle type='html'>Cloud Computing has as many interpretations as there are users, but there is one common thread among all cloud computing models and that is the convergence of information access. The &#39;Cloud&#39; holds your information, the &#39;Cloud&#39; may even compute on your information, and for real value, the &#39;Cloud&#39; may combine other sources of information to make your data, or information, more valuable.&#xa;&#xa;&lt;p&gt;This blog takes the distinct position to reason about clouds from the user&#39;s productivity perspective.&lt;/p&gt;</subtitle><link rel='http://schemas.google.com/g/2005#feed' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/posts/default'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/'/><link rel='hub' href='http://pubsubhubbub.appspot.com/'/><link rel='next' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default?start-index=26&amp;max-results=25'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><generator version='7.00' uri='http://www.blogger.com'>Blogger</generator><openSearch:totalResults>52</openSearch:totalResults><openSearch:startIndex>1</openSearch:startIndex><openSearch:itemsPerPage>25</openSearch:itemsPerPage><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-2854326382867877909</id><published>2018-09-06T09:49:00.000-07:00</published><updated>2018-09-06T09:49:07.225-07:00</updated><title type='text'>What makes posits so good?</title><content type='html'>&lt;div dir=&quot;ltr&quot; style=&quot;text-align: left;&quot; trbidi=&quot;on&quot;&gt;
  2. Computational science and all its applied forms, statistics, Deep Learning, computational finance, computational biology, etc., depend on a discrete model of equations that describe the system under study and some operator to extract information from that model. The vast majority of these operators are expressed in linear algebra form, and the dominant sub-operation in these operators is a dot product between two vectors. With the advent of big data and Deep Learning, tensor expressions have become wide-spread, but the basic operations remains squarely the dot product, as can be observed by the key operator that DL accelerators like the Google TPU implement in hardware.&lt;br /&gt;
  3. &lt;br /&gt;
  4. Let&#39;s take a closer look at the numerical properties of that dot product. Let:&lt;br /&gt;
  5. &lt;br /&gt;
  6. &lt;center&gt;
  7. &lt;i&gt;vector &lt;/i&gt;&lt;b&gt;a&lt;/b&gt;&amp;nbsp;= ( 3.2e8, 1.0, -1.0, 8.0e7)&lt;/center&gt;
  8. &lt;br /&gt;
  9. &lt;center&gt;
  10. &lt;i&gt;vector &lt;b&gt;b&amp;nbsp;&lt;/b&gt;&lt;/i&gt;= ( 4.0e7, 1.0, -1.0, -1.6e8)&lt;/center&gt;
  11. &lt;br /&gt;
  12. what would be the value of &lt;i&gt;dot&lt;/i&gt;&lt;b&gt;(a&lt;/b&gt;, &lt;b&gt;b)&lt;/b&gt;? Simple inspection confirms that &lt;i&gt;&lt;b&gt;a&lt;/b&gt;[0]&lt;b&gt;b&lt;/b&gt;[0]&lt;/i&gt; cancels &lt;i&gt;&lt;b&gt;a&lt;/b&gt;[3]&lt;b&gt;b&lt;/b&gt;[3]&lt;/i&gt; and thus the answer is &lt;b&gt;2&lt;/b&gt;.&lt;br /&gt;
  13. &lt;br /&gt;
  14. Interestingly enough, neither 32-bit floats nor 64-bit double IEEE floating point produce the correct answer when calculating this dot product in order. In contrast, even a very small posit, a&amp;nbsp;&lt;i&gt;16-bit&lt;/i&gt;&lt;b style=&quot;font-style: italic;&quot;&gt;&amp;nbsp;&lt;/b&gt;posit with 2 exponent bits, is able to produce the correct result. What gives?&lt;br /&gt;
  15. &lt;br /&gt;
  16. The individual element values of the vector are relatively ordinary numbers. But in dot products it is the dynamic range of products that the number system needs to capture, and this is where IEEE floating point fails us. IEEE floating point rounds after each product and thus the order of the computation can change the result. This is particularly troubling for concurrent systems, such as multi-core and many-core, where the programmer typically gives up control.&lt;br /&gt;
  17. &lt;br /&gt;
  18. In our particular example, the products &lt;i&gt;&lt;b&gt;a&lt;/b&gt;[0]&lt;b&gt;b&lt;/b&gt;[0]&lt;/i&gt; and &lt;i&gt;&lt;b&gt;a&lt;/b&gt;[3]&lt;b&gt;b&lt;/b&gt;[3]&lt;/i&gt; represent 1.28e16 and -1.28e16 respectively, and to be able to represent the sum of 1.28e16 + 1 requires 53bits of fraction, which 64-bit IEEE doubles do not have, creating cancellation and in the end an incorrect sum.&lt;br /&gt;
  19. &lt;br /&gt;
  20. Posits, on the other hand, have access to a &lt;i&gt;fused&lt;/i&gt; dot product, which uses a &lt;i&gt;quire&lt;/i&gt;, which can be thought of as a super accumulator that enables the computation to defer rounding till the end of the dot product. This makes it possible for a 16-bit posit to beat a 64-bit IEEE double.,&lt;br /&gt;
  21. &lt;br /&gt;
  22. In general, posits with their finer control over precision and dynamic range, present an improvement over IEEE floating point for computational science applications. In particular, business intelligence and decision support systems benefit from posits and their quires, as statistics algorithms tend to aggregate lots of data to render an assessment on which a decision depends. As the example above demonstrates, that assessment can be very wrong when using the wrong approach.&lt;/div&gt;
  23. </content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/2854326382867877909/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=2854326382867877909' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/2854326382867877909'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/2854326382867877909'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2018/09/what-makes-posits-so-good.html' title='What makes posits so good?'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-420885238688730339</id><published>2018-09-05T11:13:00.000-07:00</published><updated>2018-09-05T11:13:31.954-07:00</updated><title type='text'>Posit Number System: replacing IEEE floating point</title><content type='html'>&lt;div dir=&quot;ltr&quot; style=&quot;text-align: left;&quot; trbidi=&quot;on&quot;&gt;
  24. &lt;div style=&quot;text-align: left;&quot;&gt;
  25. The leaders in computing are now profiting from their investments in new number systems initiated half a decade ago. NVIDIA transformed the AI ecosystem with their 16-bit, half-precision float providing a boost over Intel enabling them to gain market share in the valuable data center market. Google designed the Tensor Processing Unit to accelerate AI cloud services; the TPU uses an 8-bit integer format to create a 100x benefit over its competitors. Microsoft is using an 8-bit floating point with 2-bit exponents for its Project Brainwave. And China’s Huawei is using a fixed-point format for its 4G/5G base stations to gain performance per Watt benefit over its US competitors who still use IEEE floating point.&amp;nbsp;&lt;/div&gt;
  26. &lt;div style=&quot;text-align: left;&quot;&gt;
  27. &lt;br /&gt;&lt;/div&gt;
  28. &lt;div style=&quot;text-align: left;&quot;&gt;
  29. All these companies realized that Moore’s Law and Denning’s scaling having reached a plateau, and the efficiency of computation is now a direct limiter on performance, scaling, and power. For Deep Learning specifically, and High-Performance Computing in general, IEEE floating point has shown its deficiencies in silicon efficiency, information density, and even mathematical accuracy.&lt;/div&gt;
  30. &lt;div style=&quot;text-align: left;&quot;&gt;
  31. &lt;br /&gt;&lt;/div&gt;
  32. &lt;div style=&quot;text-align: left;&quot;&gt;
  33. The posit number system is positioned as a replacement of IEEE floating point, and offer significant improvements including performance per Watt, information density, and reproducibility. The posit number system is a tapered floating point with very efficient encoding of real numbers. It has only two exceptional values; &lt;b&gt;zero&lt;/b&gt; and &lt;i&gt;&lt;b&gt;NaR&lt;/b&gt;&amp;nbsp;(not-a-real)&lt;/i&gt;. The posit encoding improves precision compared to floats of the same bit-width, which leads to higher performance and lower cost for big-data applications. Furthermore, the posit standard defines rules for reproducibility in concurrent environments enabling high-productivity and lower-cost for software application development for multi-core and many-core deployments.&lt;/div&gt;
  34. &lt;div style=&quot;text-align: left;&quot;&gt;
  35. &lt;br /&gt;&lt;/div&gt;
  36. &lt;div style=&quot;text-align: left;&quot;&gt;
  37. In the following blog posts, we will introduce the posit number system format, and report on experiments that compare IEEE floats to posits in real applications. Here is a reference to a full software environment for you tinkerers:&amp;nbsp;&lt;a href=&quot;http://stillwater-sc.com/assets/content/stillwater-universal-sw.html&quot;&gt;http://stillwater-sc.com/assets/content/stillwater-universal-sw.html&lt;/a&gt;&lt;/div&gt;
  38. &lt;/div&gt;
  39. </content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/420885238688730339/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=420885238688730339' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/420885238688730339'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/420885238688730339'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2018/09/posit-number-system-replacing-ieee.html' title='Posit Number System: replacing IEEE floating point'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-6425859006600736746</id><published>2013-01-25T12:25:00.000-08:00</published><updated>2013-01-25T12:54:57.274-08:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Business Intelligence"/><category scheme="http://www.blogger.com/atom/ns#" term="missing middle"/><category scheme="http://www.blogger.com/atom/ns#" term="Operations Research"/><category scheme="http://www.blogger.com/atom/ns#" term="optimization"/><category scheme="http://www.blogger.com/atom/ns#" term="Predictive Analytics"/><category scheme="http://www.blogger.com/atom/ns#" term="SaaS"/><category scheme="http://www.blogger.com/atom/ns#" term="supercomputing"/><title type='text'>On-Demand Supercomputing</title><content type='html'>&lt;div dir=&quot;ltr&quot; style=&quot;text-align: left;&quot; trbidi=&quot;on&quot;&gt;
  40. Except for the folks at Cray, most people are unaware of the unique requirements that set apart supercomputing infrastructure from cloud computing infrastructure. In its simplest form the difference is between latency and capacity. For business intelligence applications such as optimization and logistics many servers are required to solve a single problem, and low latency communication between the servers is instrumental for performance. The intuition behind this is easy to understand: a modern microprocessor executes 4-5 instructions per 250ps, and thus packet latencies of 10GbE, (between 5-50usec), are roughly equivalent to 100k to 1M processor instructions. If a processor is dependent on the results computed by another processor, it will have to idle till the data is available. Cumulatively, across a couple hundred servers, this can lead to peak performance that is only 1-5% of peak.&lt;br /&gt;
  41. &lt;br /&gt;
  42. Supercomputing applications are defined by these types of tightly connected concurrent processes, putting more emphasis on the performance of the interconnect, in particularly the latency. Running a traditional supercomputing application on an infrastructure designed for elastic applications, such as AWS or Azure, typically yield slow-downs by a factor 50 to 100. Measured in terms of cost, they would cost 50-100 times more to execute on a typical public cloud computing infrastructure.&lt;br /&gt;
  43. &lt;br /&gt;
  44. Most supercomputing applications are associated with very valuable economic activities of the business. As mentioned earlier, production optimization and logistics applications save companies like Exxon Mobil and Fedex billions of dollars per year. Those applications are tightly integrated in the business operation and strategic decision making of these organizations and pay for themselves many times over. However, for the SMB market these supercomputing applications offer great opportunity for revenue growth and margin improvements as well. However, their economic value is attenuated by the revenue stream they optimize; 10% improvement for a $10B revenue stream yields a $1B net benefit, but for a $10M revenue stream the benefit is just a $1M, not enough to compensate for the risk and cost that deploying a supercomputer would require.&lt;br /&gt;
  45. &lt;br /&gt;
  46. Enter On-Demand Supercomputing.&lt;br /&gt;
  47. &lt;br /&gt;
  48. In 2011, we were asked to design, construct, and deploy an On-Demand supercomputing service for a Chinese cloud vendor. The idea was to build an interconnected set of supercomputer centers in China, and offer a multi-tenant on-demand service for high-value, high-touch applications, such as logistics, digital content creation, and engineering design and optimization. The pilot program consisted of a supercomputer center in Beijing and one in Shanghai. The basic building block that was designed was a quad rack, redundant QDR IB fat-tree architecture with blade chassis at the leaves. The architecture was inspired by the observation that for the SMB market, the granularity of deployment would fall in the range of 16 to 32 processors, which would be serviced by a single chassis, keeping all communication traffic local to the chassis. The topology is shown in the following figure:&lt;br /&gt;
  49. &lt;table align=&quot;center&quot; cellpadding=&quot;0&quot; cellspacing=&quot;0&quot; class=&quot;tr-caption-container&quot; style=&quot;margin-left: auto; margin-right: auto; text-align: center;&quot;&gt;&lt;tbody&gt;
  50. &lt;tr&gt;&lt;td style=&quot;text-align: center;&quot;&gt;&lt;a href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjvBkDdMnk53e2hOHykDvbWnQpdY3e2BbJDyB_vFncz9WmBXxANoQtKaRtsaU7NKZTAG8h54m71pd_1UbseKdt-UT0mMmKnFyfyXy6cZvH5o7E64vUJL_6ZyU7DOI3CylMc5hXyg337_XH/s1600/ningbo-toplevel-infiniband-organization.png&quot; imageanchor=&quot;1&quot; style=&quot;margin-left: auto; margin-right: auto;&quot;&gt;&lt;img border=&quot;0&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjvBkDdMnk53e2hOHykDvbWnQpdY3e2BbJDyB_vFncz9WmBXxANoQtKaRtsaU7NKZTAG8h54m71pd_1UbseKdt-UT0mMmKnFyfyXy6cZvH5o7E64vUJL_6ZyU7DOI3CylMc5hXyg337_XH/s1600/ningbo-toplevel-infiniband-organization.png&quot; /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;
  51. &lt;tr&gt;&lt;td class=&quot;tr-caption&quot; style=&quot;text-align: center;&quot;&gt;Redundant QDR IB Network Topology for On-Demand Supercomputing&lt;/td&gt;&lt;/tr&gt;
  52. &lt;/tbody&gt;&lt;/table&gt;
  53. The chassis structure is easy to spot as the clusters of 20 servers at the leaves of the tree. The redundancy of the IB network is also clearly visible by the pairs of connections between all the layers in the tree. The quad configuration is a two rack symmetric setup, one pair holding one side of the redundant IB network/storage/computes. So half the quad can fall away, and the system would still have full connectivity between storage and computes. To lower the cost of the system, storage was designed around IB-based storage servers that plugged into the same infrastructure as the compute nodes. QDR throughput is balanced with PCIe gen2 and thus we were able to deliver&amp;nbsp;ephemeral blades that get their personality from the storage servers and then dynamically connect via iSCSI services to whatever storage volumes they require. This is less expensive than designing a separate NAS storage subsystem, and it gives the infrastructure flexibility to build high-performance storage solutions. It was this system that set a new world record by being the first trillion triple semantic database system leveraging a Lustre file system consisting of 8 storage servers (&lt;a href=&quot;http://stillwater-cse.blogspot.com/2011/09/trillion-triple-semantic-database.html&quot;&gt;trillion-triple-semantic-database-record&lt;/a&gt;).&lt;br /&gt;
  54. &lt;div&gt;
  55. &lt;br /&gt;&lt;/div&gt;
  56. &lt;div&gt;
  57. The provisioning of on-demand supercomputing infrastructure is bare metal, mostly to avoid any of the I/O latency degradation that virtualization injects. Given the symmetry between storage and compute and the performance offered by QDR IB, a network boot mechanism can be used to put any personality on the blades without any impact on performance. The blades have local disk for scratch space, but run their OS and data volumes off the storage servers, thus avoiding the problem of DR of state on the blades.&lt;/div&gt;
  58. &lt;div&gt;
  59. &lt;br /&gt;&lt;/div&gt;
  60. &lt;div&gt;
  61. The QDR IB infrastructure was based on Voltair switches and Mellanox HCAs.&amp;nbsp;Intel helped us tune the infrastructure, using their cluster libraries for the processors we were using, and Mellanox was instrumental in getting the IB switches in shape. Over a three week period, we went from 60% efficiency to about 94% efficiency. The full quad has a peak performance of 19.2TFlops and after tuning the infrastructure we were able to consistently deliver 18TFlops of sustained performance.&lt;/div&gt;
  62. &lt;div&gt;
  63. &lt;br /&gt;&lt;/div&gt;
  64. &lt;div&gt;
  65. The total cost of the core system was of the order of $3.6M. The On-Demand Supercomputing service offers a full dual socket server with 64GB of memory for about $5/hr, providing a cost-effective service for SMBs interested in leveraging high performance computing. For example, a digital content creation firm in Beijing leveraged about 100 servers as burst capacity for post-production. Their monthly cost to leverage a state of the art supercomputer was less than $20k per month. Similarly, a material science application was developed by a chemical manufacturer to study epitaxial growth. This allowed the manufacturer to optimize the process parameters for a thin-film process that would not have been cost-effective on a cloud infrastructure designed for elastic web applications.&lt;/div&gt;
  66. &lt;div&gt;
  67. &lt;br /&gt;&lt;/div&gt;
  68. &lt;div&gt;
  69. The take-away of this project is echoing the findings in the missing middle reports for digital manufacturing (&lt;a href=&quot;http://www.digitalmanufacturingreport.com/&quot;&gt;Digital Manufacturing Report&lt;/a&gt;). There is tremendous opportunity for SMBs to improve business operations by leveraging the same techniques as their enterprise&amp;nbsp;brethren. But the cost of commercial software for HPC is not consistent with the value provided for SMBs. Furthermore, the IT and operational skills required both to setup and manage a supercomputing infrastructure is beyond the capabilities of most SMBs. On-demand HPC services, as we have demonstrated with the supers in Beijing and Shanghai, can overcome many of these issues. Most importantly, it enables a new level of innovation by domain experts, such as professors and independent consultants, who do have the skills necessary to leverage supercomputing techniques, but up to now have not had access to public supercomputing capability and services.&lt;br /&gt;
  70. &lt;div class=&quot;separator&quot; style=&quot;clear: both; text-align: center;&quot;&gt;
  71. &lt;br /&gt;&lt;/div&gt;
  72. &lt;div class=&quot;separator&quot; style=&quot;clear: both; text-align: center;&quot;&gt;
  73. &lt;br /&gt;&lt;/div&gt;
  74. &lt;br /&gt;
  75. &lt;div class=&quot;separator&quot; style=&quot;clear: both; text-align: center;&quot;&gt;
  76. &lt;/div&gt;
  77. &lt;br /&gt;
  78. &lt;div class=&quot;separator&quot; style=&quot;clear: both; text-align: center;&quot;&gt;
  79. &lt;/div&gt;
  80. &lt;br /&gt;&lt;/div&gt;
  81. &lt;/div&gt;
  82. </content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/6425859006600736746/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=6425859006600736746' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6425859006600736746'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6425859006600736746'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2013/01/on-demand-supercomputing.html' title='On-Demand Supercomputing'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjvBkDdMnk53e2hOHykDvbWnQpdY3e2BbJDyB_vFncz9WmBXxANoQtKaRtsaU7NKZTAG8h54m71pd_1UbseKdt-UT0mMmKnFyfyXy6cZvH5o7E64vUJL_6ZyU7DOI3CylMc5hXyg337_XH/s72-c/ningbo-toplevel-infiniband-organization.png" height="72" width="72"/><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-3847916125486590159</id><published>2012-10-04T12:25:00.001-07:00</published><updated>2012-10-04T12:25:39.578-07:00</updated><title type='text'>Cloud Collaboration: Bioinformatics</title><content type='html'>&lt;div dir=&quot;ltr&quot; style=&quot;text-align: left;&quot; trbidi=&quot;on&quot;&gt;
  83. &lt;br /&gt;
  84. &lt;div style=&quot;text-align: left;&quot;&gt;
  85. &lt;span style=&quot;text-align: left;&quot;&gt;In this multi-part series, we discuss several cloud computing projects that we have completed over the course of the past 12 months. They range from experiments in computational science and engineering to full production systems for cloud service providers. This week, we&#39;ll discuss a cloud collaboration solution that was designed to bring together collaborators at different bioinformatics labs, and to connect them to an elastic cloud computing facility that offers FPGA accelerated servers, which is a facility that is increasingly important to solve the exponentially increasing compute load for bioinformatics research.&lt;/span&gt;&amp;nbsp;&lt;/div&gt;
  86. &lt;div align=&quot;JUSTIFY&quot; style=&quot;line-height: 105%; margin-bottom: 0in; orphans: 0; text-indent: 0.14in; widows: 0;&quot;&gt;
  87. &lt;/div&gt;
  88. &lt;h2 style=&quot;line-height: normal; orphans: 2; text-align: left; text-indent: 0px; widows: 2;&quot;&gt;
  89. &lt;span style=&quot;font-size: large;&quot;&gt;NEXT GENERATION BIOINFORMATICS&lt;/span&gt;&lt;/h2&gt;
  90. &lt;br /&gt;
  91. &lt;div style=&quot;text-align: left;&quot;&gt;
  92. The importance of next generation sequencers (NGS) to personalized
  93. healthcare opportunities is well documented  [1].&amp;nbsp;&lt;span style=&quot;line-height: 105%; text-align: left;&quot;&gt;Conventional medical diagnosis relies on the analysis of the
  94. patient&#39;s personal and family history, symptoms, and samples. The goal of personalized health care is to strengthen the diagnostics by including comparisons between the patient&#39;s genome and a global reference database of known disease markers.
  95. Sample testing will also be enhanced through highly sensitive
  96. measurement of adaptive immune system parameters, and molecular-level
  97. monitoring of the progression of different cancers. Outside human
  98. health, NGS will touch every aspect of our understanding of the
  99. living world, and help improve food safety, control pests, reduce
  100. pollution, find alternate sources of energy, etc.&lt;/span&gt;&lt;/div&gt;
  101. &lt;div style=&quot;text-align: left;&quot;&gt;
  102. &lt;span style=&quot;line-height: 105%; text-align: left;&quot;&gt;&lt;br /&gt;&lt;/span&gt;&lt;/div&gt;
  103. &lt;div style=&quot;text-align: left;&quot;&gt;
  104. At the lowest level, the bioinformatic questions fall into two broad
  105. classes: (i) comparing next-gen reads against reference genomes, and
  106. (ii) assembling next-gen sequences to build reference genomes.&amp;nbsp;&lt;span style=&quot;line-height: 105%; text-align: left; text-indent: 0.14in;&quot;&gt;Finding scaleable and cost-effective solutions to these questions is a major
  107. challenge. E&lt;/span&gt;&lt;span style=&quot;color: black; line-height: 105%; text-align: left; text-indent: 0.14in;&quot;&gt;ven the best algorithms need large
  108. amounts of fast and costly computer memory  (50-1024GB), and
  109. thousands of processor-hours to complete analysis on mammalian-sized
  110. genomes. Reducing execution time will improve the quality of results
  111. by allowing the analysis to be run with many different parameters.&lt;/span&gt;&lt;/div&gt;
  112. &lt;div style=&quot;text-align: left;&quot;&gt;
  113. &lt;span style=&quot;color: black; line-height: 105%; text-align: left; text-indent: 0.14in;&quot;&gt;&lt;br /&gt;&lt;/span&gt;&lt;/div&gt;
  114. &lt;div style=&quot;text-align: left;&quot;&gt;
  115. The computational landscape is changing rapidly with discovery of new
  116. algorithms, and the proliferation of reference genomes. This puts
  117. tremendous pressure on the data management and computational
  118. complexity for which a seamless solution needs to be found.&lt;/div&gt;
  119. &lt;div style=&quot;text-align: left;&quot;&gt;
  120. &lt;br /&gt;&lt;/div&gt;
  121. &lt;div style=&quot;text-align: left;&quot;&gt;
  122. Our client was a start-up that was designing a new instrument.&amp;nbsp;Our client&#39;s research instrument was able to generate up to 100GBytes of data per day, and would run continuously for several days for a single study. The raw data had to go through several stages of deep analytics to identify and correct data acquisition errors, apply data reduction and reconstruction operators, and keep track of historical performance to identify the best algorithms and to manage the research direction and software development. The algorithms required high throughput compute clusters, and the R&amp;amp;D talent that developed these algorithms was geographically dispersed throughout the US. The management and procurement of these clusters was beyond the capability or capital resources of this start-up, and on-demand cloud services could provide solutions to this CAPEX problem. However, to integrate multiple remote laboratories into a productive collaborate space required a robust and cost-effective solution to file replication so that each instrument, and all subsequent analysis results, would be readily available to the distributed team.&lt;/div&gt;
  123. &lt;h2 style=&quot;text-align: left;&quot;&gt;
  124. &lt;span style=&quot;font-size: large;&quot;&gt;NEXT GENERATION CLOUD INFRASTRUCTURE&lt;/span&gt;&lt;/h2&gt;
  125. &lt;div style=&quot;text-align: left;&quot;&gt;
  126. The core operation of the instrument was based on data acquisition of high-frequency RF signals and the use of chemicals to prepare samples. This nature&amp;nbsp;demanded that the physical machine resided in a well-controlled laboratory environment, a very different environment than a typical data center room.&amp;nbsp;The compute demands of the research algorithms were roughly in the 50TOPS range and highly latency sensitive. The cost to create that raw hardware capacity was of the order of $150k per lab, not including operational staff. The utilization of that equipment would have been very low, particularly in the beginning phases of the project when the instrument would only generate data once per month.&amp;nbsp;&amp;nbsp;Allocating the compute capacity for the research algorithms in a remote cloud solves the capex and utilization problem, but we would introduce a data movement problem. What did this trade-off look like?&lt;/div&gt;
  127. &lt;div style=&quot;text-align: left;&quot;&gt;
  128. &lt;br /&gt;&lt;/div&gt;
  129. &lt;div style=&quot;text-align: left;&quot;&gt;
  130. We evaluated two organizations:&lt;/div&gt;
  131. &lt;div style=&quot;text-align: left;&quot;&gt;
  132. &lt;/div&gt;
  133. &lt;ol style=&quot;text-align: left;&quot;&gt;
  134. &lt;li&gt;instrument and compute cluster per lab&lt;/li&gt;
  135. &lt;li&gt;instrument per lab, compute cluster in a remote cloud&lt;/li&gt;
  136. &lt;/ol&gt;
  137. &lt;br /&gt;
  138. &lt;br /&gt;
  139. The data retention was not a key attribute, and thus the cost of data storage for backups was not a factor in the design. However, link bandwidth &lt;i&gt;was &lt;/i&gt;a key attribute. The labs were at the end of relatively low bandwidth links and increasing bandwidth required a 90 day waiting period with hard caps due to infrastructure limits. The links in place were limited to 5Mb/s!!! Not very impressive, but surprisingly common. Increasing the bandwidth would have cost $30k/year extra and the 90 days waiting also made this unattractive.&lt;br /&gt;
  140. &lt;br /&gt;
  141. The capex of option 1 with two labs was about $300k with about 4 weeks of turn around time. That capex would go away in option 2, and the turn around time was reduced to days to get up and running. However, at 5Mb/s,&amp;nbsp;moving a 50GB file to a remote cloud would take several days, and worse, this cost would have to be paid every data acquisition. However, the early research instrument would take several days for a data collection, so the labs workflow was already used to having to a long latency between experiment design and data collection. But, more importantly, if the instrument takes several days to collect a large data set, if we need to migrate that data to a remote location, we want to overlap data acquisition with data transfer. Typical web application protocols don&#39;t work well in this regard, so HTTP and FTP are not attractive. The idea for the solution came from our use of Git. Git is a versioning system that is based on snapshotting a file system: that model is exactly the right technology as it is seamless and robust. This snapshotting idea lead us to Nasuni, which provides a filer that snapshots to Amazon S3 and integrates this with additional enterprise features such as volume management and volume replication&amp;nbsp;configuration.&amp;nbsp;Nasuni is relatively expensive, starting at $10k/TB/year, but the flexibility and set-and-forget feature set made it very attractive. The question was whether or not file system snapshotting would work with very low bandwidth links. If the replication performance was sufficient, then managing the persistent data volume that would determine cost would be trivial.&lt;br /&gt;
  142. &lt;h2 style=&quot;text-align: left;&quot;&gt;
  143. SYSTEM ARCHITECTURE&lt;/h2&gt;
  144. &lt;br /&gt;
  145. &lt;div style=&quot;text-align: left;&quot;&gt;
  146. &amp;nbsp;To create a robust and reliable next generation IT infrastructure, we designed and implemented a real-time distributed
  147. collaboration grid, as depicted in the figure below&lt;span style=&quot;line-height: 105%; text-align: left; text-indent: 0.14in;&quot;&gt;.&lt;/span&gt;&lt;/div&gt;
  148. &lt;div align=&quot;JUSTIFY&quot; style=&quot;line-height: 105%; margin-bottom: 0in; orphans: 0; text-indent: 0.14in; widows: 0;&quot;&gt;
  149. &lt;br /&gt;&lt;/div&gt;
  150. &lt;div class=&quot;separator&quot; style=&quot;clear: both; text-align: center;&quot;&gt;
  151. &lt;a href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioXy_91QWJYIRWs7XY8_5WaS9FePjBPmXYsmi9Ebo9VkLBrQ48B58NMWn_oZRnFHJnX6GFFMHM3TnafWr_pFPZmCzCOLhHcnIUa3N_RUAJqS5vSqw_k-EsXYv5GtRrH6a6xhhbQ7w4Mlo2/s1600/bioinformatics-service-nimbix.png&quot; imageanchor=&quot;1&quot; style=&quot;margin-left: 1em; margin-right: 1em;&quot;&gt;&lt;img border=&quot;0&quot; height=&quot;203&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioXy_91QWJYIRWs7XY8_5WaS9FePjBPmXYsmi9Ebo9VkLBrQ48B58NMWn_oZRnFHJnX6GFFMHM3TnafWr_pFPZmCzCOLhHcnIUa3N_RUAJqS5vSqw_k-EsXYv5GtRrH6a6xhhbQ7w4Mlo2/s320/bioinformatics-service-nimbix.png&quot; width=&quot;320&quot; /&gt;&lt;/a&gt;&lt;/div&gt;
  152. &lt;div align=&quot;JUSTIFY&quot; style=&quot;line-height: 105%; margin-bottom: 0in; orphans: 0; text-indent: 0.14in; widows: 0;&quot;&gt;
  153. &lt;br /&gt;&lt;/div&gt;
  154. &lt;div align=&quot;JUSTIFY&quot; style=&quot;line-height: 105%; margin-bottom: 0in; orphans: 0; text-indent: 0.14in; widows: 0;&quot;&gt;
  155. &lt;br /&gt;&lt;/div&gt;
  156. Each lab is allocated a Nasuni filer, with a sufficiently large file
  157. cache to support local working set caching. The Nasuni service creates a system where each filer becomes part of a
  158. distributed file system that is created by each filer replicating and receiving snapshots to and from the cloud storage service. The filers encrypt the
  159. snapshots and send them to the cloud store filer, which replicates
  160. the snapshots at the other locations. The cloud store lives on Amazon
  161. Web Services as an encrypted object store to provide high security
  162. and high availability. Given normal AWS S3 replication parameters,
  163. this store represents an almost perfect reliability platform. Nasuni guarantees 100% availability, which derives from the eleven-nines that Amazon offers on its S3 storage platform. (Amazon uses the term &#39;durable&#39;, which is more specific, as availability also incorporates connectivity, which Amazon can&#39;t guarantee)&lt;br /&gt;
  164. &lt;br /&gt;
  165. The filers can be configured to snapshot and replicate in a real-time
  166. fashion for small delta data changes, such as scientist work spaces.
  167. For data sets that see a lot of change, such as the output of data collection instruments, it is more appropriate to snapshot at lower frequencies
  168. to create good traffic attributes during replication. Furthermore,
  169. since our client&#39;s data sets are generated over multiple days, it is not
  170. useful to replicate these data sets in real-time as the data is not
  171. complete and the overhead is too high.&lt;br /&gt;
  172. &lt;br /&gt;We also allocated a filer at the high performance computing cloud
  173. provider. The role of this filer is different from the filers at the
  174. laboratories. Whereas the filers at the laboratories function as a
  175. buffer for the data generation instruments, the filer at the compute
  176. cloud functions as an aggregation and working set manager. At the
  177. compute cloud, bioinformatic applications are run that use all, or large
  178. subsets of the data, and produce large, new sets of results. These
  179. results need to be replicated to the laboratory filers. The
  180. aggregation data can be held in a big data store, such as Hadoop or
  181. Riak, so that the CSP filer snapshot cache is not over-committed
  182. during these deep analytics compute cycles.&lt;br /&gt;
  183. &lt;h2 style=&quot;text-align: left;&quot;&gt;
  184. Conclusions&lt;/h2&gt;
  185. &lt;div&gt;
  186. To test the data replication performance, we used a 60GB data set that was generated by the instrument over a two day period. We used a volume snapshot rate of three times a day, and we throttled the filers bandwidth to only 4Mb/s so that we would not severely impact normal Internet traffic. At our Cloud Service Provider, we ran a VMware-hosted filer as the target. The Nasuni filer received our data set in roughly 35 hours. This experiment demonstrated that even with a high volume instrument the data movement delay to a CSP was not a significant source of delay. When the instrument would improve it data acquisition rates the link rates could be improved to match this increased performance.&amp;nbsp;&lt;/div&gt;
  187. &lt;div&gt;
  188. &lt;br /&gt;&lt;/div&gt;
  189. &lt;div&gt;
  190. Once we had proven that data movement using the Nasuni filers and storage service had reasonable performance even on very low bandwidth links, the additional features provided by the Nasuni storage service made this solution very attractive. IT personal can configure the filers and the volume parameters, and manage these against the IT resources available, independent from the research teams. Those research teams are presented with effectively an infinite storage resource and complete flexibility to compute locally, or in the cloud, all the while knowing that whatever results they produce, the data is visible to all collaborators without having to manually engage upload/download workflows.&lt;/div&gt;
  191. &lt;div&gt;
  192. &lt;br /&gt;&lt;/div&gt;
  193. &lt;div&gt;
  194. The one attribute that is of concern with this cloud collaboration architecture is the cost of storage. At higher storage capacity the Nasuni service drops down in cost significantly, but the assumption behind the Nasuni service is that all data is enterprise critical. In research oriented environments, data sets tend to grow rapidly and unbounded, but none of it is enterprise critical. To not break the bank, we must introduce active storage management. Once data has been analyzed and transformed, it should be copied to local NAS filers, and removed from the Nasuni service. This provides a solution that controls cost, and as a positive side effect, it keeps the data sets actively managed.&lt;/div&gt;
  195. &lt;div&gt;
  196. &lt;br /&gt;&lt;/div&gt;
  197. &lt;div&gt;
  198. The end result of this project was that we saved the client $300k in capex and the overhead of managing two hardware clusters. The clusters would have been essential for creating the value-proposition of the client&#39;s innovation, but&amp;nbsp;once the core analytics algorithm was developed,&amp;nbsp;would have gone unused after the first year. The high data expansion volume of the instrument made a cloud computing solution non-trivial, but using the Nasuni storage service provided a set-and-forget solution to the data distribution problem.&lt;/div&gt;
  199. &lt;br /&gt;
  200. &lt;h2 style=&quot;text-align: left;&quot;&gt;
  201. References&lt;/h2&gt;
  202. &lt;ol&gt;
  203. &lt;li&gt;&lt;div class=&quot;western&quot; style=&quot;margin-bottom: 0in;&quot;&gt;
  204. &lt;span style=&quot;font-size: x-small;&quot;&gt;A.
  205. Kahvejian, J. Quackenbush, and J.F. Thompson, &lt;i&gt;What
  206. would you do if you could sequence everything?, &lt;/i&gt;&lt;b&gt;Nature
  207. Biotechnology&lt;/b&gt;,
  208. 2008, Vol. 26, pp
  209. 1125-1133,&amp;nbsp;&lt;a class=&quot;western&quot; href=&quot;http://www.nature.com/nbt/journal/v26/n10/full/nbt1494.html&quot;&gt;&lt;span style=&quot;font-family: Times New Roman, serif;&quot;&gt;http://www.nature.com/nbt/journal/v26/n10/full/nbt1494.html&lt;/span&gt;&lt;/a&gt;&lt;/span&gt;&lt;/div&gt;
  210. &lt;/li&gt;
  211. &lt;/ol&gt;
  212. &lt;br /&gt;
  213.  
  214.  
  215. &lt;/div&gt;
  216. </content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/3847916125486590159/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=3847916125486590159' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3847916125486590159'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3847916125486590159'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2012/10/cloud-collaboration-bioinformatics.html' title='Cloud Collaboration: Bioinformatics'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioXy_91QWJYIRWs7XY8_5WaS9FePjBPmXYsmi9Ebo9VkLBrQ48B58NMWn_oZRnFHJnX6GFFMHM3TnafWr_pFPZmCzCOLhHcnIUa3N_RUAJqS5vSqw_k-EsXYv5GtRrH6a6xhhbQ7w4Mlo2/s72-c/bioinformatics-service-nimbix.png" height="72" width="72"/><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-3358331899338056895</id><published>2012-10-03T07:10:00.000-07:00</published><updated>2012-10-03T07:10:30.381-07:00</updated><title type='text'>Cloud Collaboration</title><content type='html'>&lt;div dir=&quot;ltr&quot; style=&quot;text-align: left;&quot; trbidi=&quot;on&quot;&gt;
  217. The past 12 months, we have implemented a handful of global cloud platforms that connect US, EU, and APAC. The common impetus behind these projects is to connect brain trusts in these geographies. Whether they are supply chains in Asia program managed from the EU, healthcare cost improvements in the US by using radiologists in India, or high-tech design teams that are collaborating on a new car or smart phone design, all these efforts are trying to implement the IT platform to create the global village.&lt;br /&gt;
  218. &lt;br /&gt;
  219. The teachings provided by these implementations are that cloud computing is more or less a solved problem, but cloud collaboration is far from done. Cloud collaboration from an architecture point of view is similar to the constraints faced by mobile application platforms, so there is no doubt that in the next couple of years we&#39;ll see lots of nascent solutions to the fundamental problem of mobility and cloud collaboration: data movement.&lt;br /&gt;
  220. &lt;br /&gt;
  221. The data sets in our US-China project measured in the range from tens to hundreds of TBytes, but data expansion was modest at a couple of GBytes a day. For a medical cloud computing project, the data set was more modest at 35TBytes, but the data expansion of these data sets could be as high as 100GB per day, fueled by high volume instruments, such as MRI or NGS machines. In the US-China collaboration, the problem was network latency and packet loss, whereas in the medical cloud computing project, the problem was how to deal with multi-site high-volume data expansions. The cloud computing aspect of all these projects was literally less than a couple of man weeks worth of work. The cloud collaboration aspect of these projects all required completely new technology developments.&lt;br /&gt;
  222. &lt;br /&gt;
  223. In the next few weeks, I&#39;ll describe the different projects, their business requirements, their IT architecture&amp;nbsp;manifestation, and the key technologies that we had to develop to deliver their business value.&lt;br /&gt;
  224. &lt;br /&gt;
  225. &lt;br /&gt;&lt;/div&gt;
  226. </content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/3358331899338056895/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=3358331899338056895' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3358331899338056895'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3358331899338056895'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2012/10/cloud-collaboration.html' title='Cloud Collaboration'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-6837789627720561138</id><published>2011-09-29T13:09:00.000-07:00</published><updated>2011-09-29T13:17:09.952-07:00</updated><title type='text'>Amazon Silk: split browser architecture</title><content type='html'>&lt;div dir=&quot;ltr&quot; style=&quot;text-align: left;&quot; trbidi=&quot;on&quot;&gt;
  227. &lt;a href=&quot;http://amazonsilk.wordpress.com/2011/09/28/introducing-amazon-silk/&quot;&gt;Amazon Silk&lt;/a&gt;&lt;br /&gt;
  228. &lt;br /&gt;
  229. Content Delivery Networks, and WAN optimization, provided a generic acceleration solution to get common content closer to the client device, but on mobile devices the delivery performance of the last mile was still a problem. Many websites still do not have mobile optimized content, and sucking down a 3Mpixel JPG and render it on a 320x240 pixel display is just plain wrong. With the introduction of Amazon Silk, which uses the cloud to aggregate, cache, precompile, and predict, the client-side experience can now be optimized for the device that everybody glamours for: the tablet.&lt;br /&gt;
  230. &lt;br /&gt;
  231. This is going to create an even bigger disconnect between the consumer IT experience and the enterprise IT experience. On the Amazon Fire you will be able to pull up, nearly&amp;nbsp;instantaneously, common TV video clips and connect to millions of books. But most enterprises will find it difficult to invest in WAN optimization gear that would replicate that experience on the corporate network for your day to day work.&lt;br /&gt;
  232. &lt;br /&gt;
  233. Amazon Silk is another example of the power that the cloud provides for doing heavy computes and caching that enables low-capability devices to roam.&lt;/div&gt;
  234. </content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/6837789627720561138/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=6837789627720561138' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6837789627720561138'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6837789627720561138'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2011/09/amazon-silk-split-browser-architecture.html' title='Amazon Silk: split browser architecture'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-2966001777820741945</id><published>2011-09-14T14:02:00.000-07:00</published><updated>2011-09-14T14:02:18.925-07:00</updated><title type='text'>Trillion Triple Semantic Database</title><content type='html'>&lt;div dir=&quot;ltr&quot; style=&quot;text-align: left;&quot; trbidi=&quot;on&quot;&gt;
  235. The Semantic Web captures the semantics, or meaning, of data, and machines are enabled to interact with that meta data. It is an idea of WWW pioneer Tim Berners-Lee who observed that although search engines index much of the Web&#39;s content, keywords can only provide an indirect association to the meaning of the article&#39;s content. He foresees a number of ways in which developers and authors can create and use the semantic web to help context-understanding programs to better serve knowledge discovery.
  236. &lt;br /&gt;
  237. &lt;br /&gt;
  238. Tim Berners-Lee originally expressed the vision of the Semantic Web as follows:&lt;br /&gt;
  239. &lt;blockquote&gt;
  240. &lt;i&gt;
  241. I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web – the content, links, and transactions between people and computers. A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The intelligent agents people have touted for ages will finally materialize.&lt;/i&gt;
  242. &lt;/blockquote&gt;
  243. &lt;br /&gt;
  244. The world of semantic databases just got a little bit more interesting with the announcement by Franz, Inc. and Stillwater SC of having reached a trillion triple semantic data store for telecommunication data.&lt;br /&gt;
  245. &lt;br /&gt;
  246. &lt;a href=&quot;http://www.franz.com/about/press_room/trillion-triples.lhtml&quot;&gt;http://www.franz.com/about/press_room/trillion-triples.lhtml&lt;/a&gt;
  248. &lt;br /&gt;
  249. &lt;br /&gt;
  250. The database was constructed with an HPC on-demand cloud service and occupied 8 compute servers and 8 storage servers. The compute servers contained dual socket Xeons with 64GB of memory connecting through an QDR IB network to a 300TB SAN. The trillion triple data set spanned roughly 100TB of storage. It took roughly two weeks to load the data, but after that database provided interactive query rates for knowledge discovery and data mining.&lt;br /&gt;
  251. &lt;br /&gt;
  252. The gear on which this result was produced is traditional HPC gear that emphasizes scalability and low latency interconnect. As a comparison, a billion triple version of the database was created on Amazon Web Services but the performance was roughly 3-5x slower. To create a trillion triple semantic database on AWS would have cost $75k and would have taken 6 weeks to complete.&lt;/div&gt;
  253. </content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/2966001777820741945/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=2966001777820741945' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/2966001777820741945'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/2966001777820741945'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2011/09/trillion-triple-semantic-database.html' title='Trillion Triple Semantic Database'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-4693795286901305432</id><published>2011-07-04T12:30:00.001-07:00</published><updated>2011-07-04T12:59:30.862-07:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="&quot;cloud computing&quot;"/><category scheme="http://www.blogger.com/atom/ns#" term="&quot;provisioning stacks&quot;"/><title type='text'>What would you do with infinite computes?</title><content type='html'>Firing up a 1000 processor deep analytics cluster in the cloud to solve a market segmentation question regarding your customer orders during Christmas 2010, or a sentiment analysis of your company&#39;s facebook fan page now costs less than having lunch in Palo Alto. &lt;br /&gt;&lt;br /&gt;The cloud effectively provides infinite computes, and to some degree infinite storage, although the costs of non-ephemeral storage might murk that analogy up a bit. So what would you do differently now you have access to a global supercomputer?&lt;br /&gt;&lt;br /&gt;When I pose this question to my clients, it quickly reveals that their business processes are ill-prepared to take advantage of this opportunity. We are roughly half a decade into the cloud revolution, and at least a decade into the &#39;competing on analytics&#39; mind set, but the typical enterprise IT shop is still unable to make a difference in the cloud. &lt;br /&gt;&lt;br /&gt;However, change may be near. Given the state of functionality in software stacks like RightScale and Enstratus we might see a discontinuity in this inability to take advantage of the cloud. These stacks are getting to the point that an IT novice is able to provision complex applications into the cloud. Supported by solid open source provisioning stacks like Eucalyptus and Cloud.com, building reliable and adaptive software service stacks in the cloud is becoming child&#39;s play. &lt;br /&gt;&lt;br /&gt;What I like about these environment is that they are cloud agnostic. For proper DR/BPC a single cloud provider would be a single point of failure and thus a non-starter. But these tools make it possible to run a live application across multiple cloud vendors thus solving the productivity and agility requirements that come with the territory of an Internet application.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/4693795286901305432/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=4693795286901305432' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/4693795286901305432'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/4693795286901305432'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2011/07/what-would-you-do-with-infinite.html' title='What would you do with infinite computes?'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-3956905664075583289</id><published>2010-11-27T13:37:00.001-08:00</published><updated>2010-11-27T14:49:09.456-08:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="silicon innovation"/><category scheme="http://www.blogger.com/atom/ns#" term="start-ups"/><category scheme="http://www.blogger.com/atom/ns#" term="venture capital"/><title type='text'>Why is there so little innovation in cloud hardware?</title><content type='html'>With the explosion of data and the need to make sense out of it all on a smart phone is creating an interesting opportunity. Mobile devices need high performance at low power, and Apple seems to be the only one that has figured out that having your own processor team and IP is actually a key advantage. And the telcos will need Petascale data centers to manage content, knowledge management, and operational intelligence and the performance per Watt of general purpose CPUs from IBM, Intel, and AMD are at least two order of magnitude away from what is possible. So why is there so little innovation in cloud hardware?&lt;br /&gt;&lt;br /&gt;The rule of thumb for creating a new chip venture is $50-75M. Clearly the model where your project is just an aggregation of third party IP blocks is not a very interesting investment as it would create no defendable position in the market place. So from a differentiation point of view early stage chip companies need to have some unique IP. And this IP needs to be substantial. This creates the people and tool cost that makes chip design expensive. &lt;br /&gt;&lt;br /&gt;Secondly, to differentiate on performance, power, or space you have to be at least closer to the leading edge. When Intel is at 32nm, don’t pick 90nm as a feasible technology. So mask costs are measured in the millions for products that try to compete in high-value silicon. &lt;br /&gt;&lt;br /&gt;Thirdly, it takes at least two product cycles to move the value chain. Dell doesn’t move until it can sell 100k units a month, and ISVs don’t move until there millions of units of installed base. So the source of the $50M-$75M needed for fabless semi is that creating new IP is a $20-25M problem if presented to the market as a chip and it takes two cycles to move the supply chain, and it takes three cycles to move the software.&lt;br /&gt;&lt;br /&gt;The market dynamics of IT has created this situation. It used to be the case that the enterprise market drove silicon innovation. However, the enterprise market is now dragging the silicon investment market down. Enterprise hardware and software is no longer the driving force: the innovation is now driven by the consumer market. And that game is played and controlled by the high volume OEMs. Secondly, their cost constraints and margins make delivering IP to these OEMs very unattractive: they hold all the cards and attenuate pricing so that continued engineering innovation is hard to sustain for a startup. Secondly, an OEM is not interested in creating unique IP by a third party: it would deleverage them. So you end up getting only the non-differentiable pieces of technology and a race to the bottom.&lt;br /&gt;&lt;br /&gt;Personally however, I believe that there is a third wave of silicon innovation brewing. When I calculate the efficiency that Intel gets out of a square millimeter of silicon and compare that to what is possible I see a thousand fold difference. So, there are tremendous innovation possibilities from an efficiency point of view alone. Combining it with the trend to put intelligence into every widget and connecting them wirelessly provides the application space where efficient silicon that delivers high performance per Watt can really shine AND have a large potential market. Mixed-signal and new processor architectures will be the differentiators and the capital markets will at one point recognize the tremendous opportunities present to create a next generation technology that creates these intelligent platforms.&lt;br /&gt;&lt;br /&gt;Until then, us folks that are pushing the envelope will continue to refine our technologies so we can be ready when the capital market catches up with the opportunities.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/3956905664075583289/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=3956905664075583289' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3956905664075583289'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3956905664075583289'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2010/11/why-is-there-so-little-innovation-in.html' title='Why is there so little innovation in cloud hardware?'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-5122310104319664773</id><published>2010-07-19T15:34:00.000-07:00</published><updated>2010-07-19T15:53:40.347-07:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="cloud computing standards"/><title type='text'>OpenStack: potential for a cloud standard</title><content type='html'>Today, Rackspace open sourced its cloud platform and announced to create a collaborative effort that includes NASA, Citrix, and Dell to build an open source cloud platform, dubbed &lt;a href=&quot;http://www.openstack.org/press/rackspace-openstack-7-19-2010/&quot;&gt;OpenStack&lt;/a&gt;. Finally, the world of cloud computing gets some weight behind a potential standard. Google, Microsoft, Amazon, and big SaaS players like Salesforce.com and Netsuite are getting too isolated and too powerful to be believable to drive any type of standard for interoperability in the cloud. An open source stack could really break this open.&lt;br /&gt;&lt;br /&gt;If Dell is true to its word to distribute OpenStack with its storage and server products, and Citrix is true to its word to drive OpenStack into their customer base, this effort has some real power players behind it that could provide the counter weight needed to stop the economic lock-in of the pioneers. &lt;br /&gt;&lt;br /&gt;This announcement is very powerful as it provides a platform to accelerate innovation particularly from the university research where long-tailed projects simply do not get the time of day from Google or Microsoft. By offering a path to get integrated in an open source cloud platform, applications and run-times for genetics and proteomics, and deep computational engineering problems like material science and molecular dynamics get an opportunity to leverage cloud computing as a collective.&lt;br /&gt;&lt;br /&gt;Most of the innovation in these verticals is delivered through open source and university research. By building solutions into a collective, the university research groups can build momentum that they could &lt;span style=&quot;font-style:italic;&quot;&gt;not &lt;/span&gt; build with adopting solutions from commercial vendors like Amazon AWS, Google Apps, or Microsoft Azure. &lt;br /&gt;&lt;br /&gt;It also opens up great innovation opportunities for university IT shops that have to manage clusters themselves. Grid computing has proven to be very complicated and heavy-handed for these IT teams, but hopefully an effort like OpenStack with backing from Rackspace, NASA, Dell, and Citrix can give these teams a shot in the arm. The university clusters can be run with utmost efficiency and tailored to the workload set of the university, and OpenStack gear at Rackspace or participating data centers can be used to deal with demand spikes without any modification to the cloud applications. &lt;br /&gt;&lt;br /&gt;These types of problems will always exists and only a cloud computing standard will be able to smooth the landscape. Let&#39;s hope that OpenStack with its backers can be the first step towards that level playing field.&lt;br /&gt;&lt;br /&gt;This is exciting news for cloud computing developers and users.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/5122310104319664773/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=5122310104319664773' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5122310104319664773'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5122310104319664773'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2010/07/openstack-potential-for-cloud-standard.html' title='OpenStack: potential for a cloud standard'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-5879169206350054371</id><published>2010-07-16T11:25:00.000-07:00</published><updated>2010-07-16T11:51:55.057-07:00</updated><title type='text'>The Intercloud</title><content type='html'>Found a wonderful post by Greg Papadopoulos in which he postulates the trend towards &lt;a href=&quot;http://blogs.sun.com/Gregp/entry/the_intercloud&quot;&gt;interclouds&lt;/a&gt;. Greg argues that Amazon&#39;s AWS BYOS/IaaS (Bring Your Own Stack) is the perfect marriage of simplicity and functionality that it will be with us for a long time. SaaS is the new delivery norm of software, and PaaS is the needed productivity layer to hide the complexity of IaaS. The proliferation of SaaS on top of PaaS on top of IaaS is the wrath of early technology adoption when most of the functionality is still in its infancy.&lt;br /&gt;&lt;br /&gt;As Greg writes: &lt;i&gt;&lt;br /&gt;&quot;Productive and in-production are different concepts, however. And as much as AWS seems to have found the lowest common denominator on the former with IaaS, how at-scale production will actually unfold will be a watershed for the computing industry. &lt;br /&gt;&lt;br /&gt;Getting deployed and in production raises an incredible array of concerns that the developer doesn&#39;t see. The best analogy here is to operating systems; basic sub-systems like scheduling, virtual memory and network and storage stacks are secondary concerns to most developers, but are primary to the operator/deployer who&#39;s job it is to keep the system running well at a predictable level of service.&lt;br /&gt;&lt;br /&gt;Now layer on top of this information security, user/service identity, accounting and audit, and then do this for hundreds or thousands of applications simultaneously and you begin to see why it isn&#39;t so easy. You also begin to see why people get twitchy about the who, where, and how of their computing plant.&lt;br /&gt;&lt;br /&gt;Make no mistake, I have no doubt that cloud (nee network, grid) computing will become the organizing principle for public and private infrastructure. The production question is what the balance will be.  Which cloud approach will ultimately win?  Will it be big public utility-like things, or more purpose-built private enterprise ones?&lt;br /&gt;&lt;br /&gt;The answer: yes. There will be no more of a consolidation to a single cloud than there is to a single network. &quot;&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;He goes on to say the cloud will organize much like the energy business with a handful of very large networks supported by hundreds of regional and national companies. In this comparison, Greg finds an analogy in the internetworking development. Connecting all these federated entities together has created tremendous value, and thus it is reasonable to expect that the cloud will organize as a federated system as well. But to accomplish that, the cloud community needs to develop the right standards, just like the Internet community did for internetworking so that nobody has to be afraid to become an isolated island.&lt;br /&gt;&lt;br /&gt;Putting on my developer&#39;s hat, this fear of becoming isolated is what is holding me back to commit to Google Apps or Microsoft Azure: they feel too proprietary in this age of open source and federated clouds. One of my core requirements for cloud applications is that the application could migrate from private cloud to public cloud and vice versa. When the economic value of the application goes up or down I want to be able to allocate it on the right infrastructure to maximize profit or minimize cost. Closed environments like Salesforce.com, Google Apps, or Microsoft Azure are a concern as these environments create a fierce lock-in for my application. Encapsulating it in an OVF virtual appliance provides me much greater flexibility at the cost of not having productive elastic computing services. That capability is maturing as we speak as its value is as obvious as it is needed.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/5879169206350054371/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=5879169206350054371' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5879169206350054371'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5879169206350054371'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2010/07/intercloud.html' title='The Intercloud'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-5961073884696043584</id><published>2010-07-14T10:02:00.000-07:00</published><updated>2010-07-14T10:12:24.929-07:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="amazon"/><category scheme="http://www.blogger.com/atom/ns#" term="AWS"/><category scheme="http://www.blogger.com/atom/ns#" term="utility computing"/><title type='text'>Amazon IT moves into AWS</title><content type='html'>&lt;a href=&quot;http://searchcloudcomputing.techtarget.com/news/article/0,289142,sid201_gci1516269,00.html&quot;&gt;Amazon.com attempts IT switch to cloud computing&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Amazon&#39;s e-commerce site is planning to move into Amazon Web Services. Jen Boden is Amazon&#39;s e-commerce IT director. &lt;br /&gt;&lt;br /&gt;&lt;i&gt;&quot;Boden said her organization is in the preliminary stages of moving into AWS -- she started with some simple, homegrown applications, such as a list maintained for HR, which her team moved to AWS successfully. Larger sections of IT operations will move later with the financials likely to be last, since they are the most sensitive to security and compliance needs. Planning began last year, and the whole process might take another year and a half.&quot;&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;From the article it is interesting to see the confirmation that Amazon&#39;s own enterprise IT needs to go through the same transformation as any other enterprise IT team that has decades of old applications and data silos to support. &lt;br /&gt;&lt;br /&gt;This effort is a great shot in the arm for cloud computing. The past year, most enterprise class IT shops have started pilot programs to figure out how to incorporate cloud storage and cloud computing into their roadmaps. When leaders like Amazon can point to solutions that others can follow, the laggards will come on board and we can finally move to the next phase in cloud computing and that is standards. With standards, utility computing will come one step closer.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/5961073884696043584/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=5961073884696043584' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5961073884696043584'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5961073884696043584'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2010/07/amazon-it-moves-into-aws.html' title='Amazon IT moves into AWS'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-4899412106043284642</id><published>2010-07-11T08:13:00.000-07:00</published><updated>2010-07-11T08:39:27.216-07:00</updated><title type='text'>Complexity and fault-tolerance</title><content type='html'>As an engineer I frequently look towards biology to get inspired or get ideas how complex systems need to be put together to stand the test of time. In this quest, I came across a wonderful &lt;a href=&quot;http://opa.yale.edu/news/article.aspx?id=7508&quot;&gt;article&lt;/a&gt; from a Yale research team that compared the transcriptional regulatory network of a bacterium to the call graph of the Linux operating system.&lt;br /&gt;&lt;br /&gt;&lt;a onblur=&quot;try {parent.deselectBloggerImageGracefully();} catch(e) {}&quot; href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgElLvojfN9-5yr8GBx8UTt3RmVoliola64VjfouR5t54rtQhwJJGnn7IwFhAUI-OgSdRgbFsxJe7Ju-MOxPNokdNrbL85gUnSjHwmAUlBemv7BXxlOuSeWk77S3iEriiAo6BndQfpoTiwD/s1600/rna-vs-linux.jpg&quot;&gt;&lt;img style=&quot;float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 314px; height: 200px;&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgElLvojfN9-5yr8GBx8UTt3RmVoliola64VjfouR5t54rtQhwJJGnn7IwFhAUI-OgSdRgbFsxJe7Ju-MOxPNokdNrbL85gUnSjHwmAUlBemv7BXxlOuSeWk77S3iEriiAo6BndQfpoTiwD/s400/rna-vs-linux.jpg&quot; border=&quot;0&quot; alt=&quot;&quot;id=&quot;BLOGGER_PHOTO_ID_5492669811403384034&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&lt;span style=&quot;font-style:italic;&quot;&gt;“It is a commonplace metaphor that the genome is the operating system of a living organism. We wanted to see if the analogy actually holds up,” said Mark Gerstein, the Albert L. Williams Professor of Biomedical Informatics; professor of molecular biophysics and biochemistry, and computer science; and senior author of the paper.&lt;br /&gt;&lt;br /&gt;Both E coli and the Linux networks are arranged in hierarchies, but with some notable differences in how they achieve operational efficiencies. The molecular networks in the bacteria are arranged in a pyramid, with a limited number of master regulatory genes at the top that control a broad base of specialized functions, which act independently.&lt;br /&gt;&lt;br /&gt;In contrast, the Linux operating system is organized more like an inverted pyramid, with many different top-level routines controlling few generic functions at the bottom of the network. Gerstein said that this organization arises because software engineers tend to save money and time by building upon existing routines rather than starting systems from scratch.&lt;br /&gt;&lt;br /&gt;“But it also means the operating system is more vulnerable to breakdowns because even simple updates to a generic routine can be very disruptive,” Gerstein said. To compensate, these generic components have to be continually fine-tuned by designers.&lt;br /&gt;&lt;br /&gt;Operating systems are like urban streets – engineers tend to focus on areas that get a lot of traffic,” said Gerstein. “We can do this because we are designing these changes intelligently.”&lt;/span&gt; &lt;br /&gt;&lt;br /&gt;As an engineer this is a very recognizable failure mode of functional design by humans. We are so focused on providing the functionality at the lowest cost possible that the system wide aspects of how the functionality should be reliably provided is lost. A biological system like the human body could lose an eye, a digit, or even a limb and we would still be able to function as a human being. But if you take one leg of a table or chair, the table or chair ceases to function as designed. It is only in high exposure or non-serviceable designs that this broader context is designed into the functionality of the system. Control systems of nuclear plants, or trading algorithms in finance are examples, as are the operating systems of deep-space vehicles. &lt;br /&gt;&lt;br /&gt;Cloud computing today needs some innovations to address the broader system exposure of a business process executing in a remote location. For example, security and authenticity of data or a software asset are the two elements that become more nebulous in a cloud context. But solutions to these problems exist and cloud computing will adopt these as best practices. Cloud computing will become a better implementation of what private companies do with their internal IT today. Particularly at the SMB level, cloud computing is already much stronger than most internal IT processes, with well defined disaster recovery processes and geographical redundancy: two elements that are beyond the capital reach of most SMBs. Cloud computing is shaping up to be a new organization of IT capability that will enable the next generation of business process innovation.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/4899412106043284642/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=4899412106043284642' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/4899412106043284642'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/4899412106043284642'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2010/07/complexity-and-fault-tolerance.html' title='Complexity and fault-tolerance'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgElLvojfN9-5yr8GBx8UTt3RmVoliola64VjfouR5t54rtQhwJJGnn7IwFhAUI-OgSdRgbFsxJe7Ju-MOxPNokdNrbL85gUnSjHwmAUlBemv7BXxlOuSeWk77S3iEriiAo6BndQfpoTiwD/s72-c/rna-vs-linux.jpg" height="72" width="72"/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-6074144264837829857</id><published>2010-01-25T07:31:00.000-08:00</published><updated>2010-01-25T07:49:17.224-08:00</updated><title type='text'>Governmental data</title><content type='html'>With governmental data being pushed into the public cloud, APIs to access them are rapidly proliferating. The Guardian has set up a &lt;a href=&quot;http://www.guardian.co.uk/world-government-data&quot;&gt;portal&lt;/a&gt; that allows users to search through the governmental data of the UK, US, Australia, and New Zealand and see how countries compare. There is also an active quest for good visualizations and the portal solicits its users to suggest visualizations. &lt;br /&gt;&lt;br /&gt;One that caught my eye is called &lt;a href=&quot;http://www.informationisbeautiful.net/visualizations/the-billion-dollar-gram/&quot;&gt;the-billion-dollar-gram&lt;/a&gt; which visualizes the relative budget size of programs and economic activity, with some interesting cross country and industry comparisons. &lt;br /&gt;&lt;a onblur=&quot;try {parent.deselectBloggerImageGracefully();} catch(e) {}&quot; href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_0bOO1vKBf39Shjl1LT-pSJ_9mjRfCdz8GdQl25cP6VIr1ntjxL1uyeNn9c5_pmsz5KmN-xFB78Woa6e3Q31CjdeeqKAmaq1FbdRH5nWxFPm57FHa4Cb-N8xXABH2Rtogm1Yf6BpjJpQK/s1600-h/billion_dollar_960.gif&quot;&gt;&lt;img style=&quot;display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 272px; height: 400px;&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_0bOO1vKBf39Shjl1LT-pSJ_9mjRfCdz8GdQl25cP6VIr1ntjxL1uyeNn9c5_pmsz5KmN-xFB78Woa6e3Q31CjdeeqKAmaq1FbdRH5nWxFPm57FHa4Cb-N8xXABH2Rtogm1Yf6BpjJpQK/s400/billion_dollar_960.gif&quot; border=&quot;0&quot; alt=&quot;&quot;id=&quot;BLOGGER_PHOTO_ID_5430702398401196018&quot; /&gt;&lt;/a&gt;&lt;br /&gt;The ease with which this data can now be gathered by simple REST calls is making it possible to cross correlate this data with private enterprise data. The initial difficulties to overcome of course are data quality and the Guardian developers understood that. They are actively scrubbing the data with the help of their early adopters. From a statisticians or academicians point of view the data legends are missing making it difficult to properly compare these data sets so expect lots of misinformation to be generated by the early uses of these public portals. Nonetheless, we have to start somewhere and a transparent system that makes it easy to access lots of data is the best first start.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/6074144264837829857/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=6074144264837829857' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6074144264837829857'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6074144264837829857'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2010/01/governmental-data.html' title='Governmental data'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_0bOO1vKBf39Shjl1LT-pSJ_9mjRfCdz8GdQl25cP6VIr1ntjxL1uyeNn9c5_pmsz5KmN-xFB78Woa6e3Q31CjdeeqKAmaq1FbdRH5nWxFPm57FHa4Cb-N8xXABH2Rtogm1Yf6BpjJpQK/s72-c/billion_dollar_960.gif" height="72" width="72"/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-5344525430707798107</id><published>2009-12-02T15:11:00.000-08:00</published><updated>2009-12-02T15:14:43.504-08:00</updated><title type='text'>Government as a Platform</title><content type='html'>I saw a tweet fly by from Tim O&#39;Reilly with the above label. When googling the matter, I came across a presentation Tim gave this summer exploring the new universe of APIs that enable access to governmental data. Combining this data with operational pieces in a typical CRM or ERP database would constitute a powerful means for ERP optimization.&lt;br /&gt;&lt;br /&gt;Here is the presentation:&lt;div style=&quot;width:425px;text-align:left&quot; id=&quot;__ss_1537624&quot;&gt;&lt;a style=&quot;font:14px Helvetica,Arial,Sans-serif;display:block;margin:12px 0 3px 0;text-decoration:underline;&quot; href=&quot;http://www.slideshare.net/timoreilly/government-as-platform&quot; title=&quot;Government As Platform&quot;&gt;Government As Platform&lt;/a&gt;&lt;object style=&quot;margin:0px&quot; width=&quot;425&quot; height=&quot;355&quot;&gt;&lt;param name=&quot;movie&quot; value=&quot;http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=govplatform-090605065618-phpapp02&amp;stripped_title=government-as-platform&quot; /&gt;&lt;param name=&quot;allowFullScreen&quot; value=&quot;true&quot;/&gt;&lt;param name=&quot;allowScriptAccess&quot; value=&quot;always&quot;/&gt;&lt;embed src=&quot;http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=govplatform-090605065618-phpapp02&amp;stripped_title=government-as-platform&quot; type=&quot;application/x-shockwave-flash&quot; allowscriptaccess=&quot;always&quot; allowfullscreen=&quot;true&quot; width=&quot;425&quot; height=&quot;355&quot;&gt;&lt;/embed&gt;&lt;/object&gt;&lt;div style=&quot;font-size:11px;font-family:tahoma,arial;height:26px;padding-top:2px;&quot;&gt;View more &lt;a style=&quot;text-decoration:underline;&quot; href=&quot;http://www.slideshare.net/&quot;&gt;documents&lt;/a&gt; from &lt;a style=&quot;text-decoration:underline;&quot; href=&quot;http://www.slideshare.net/timoreilly&quot;&gt;Tim O&amp;rsquo;Reilly&lt;/a&gt;.&lt;/div&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;One of the tidbits of info that jumped out for me was that 45% of all map mashups are based on Google maps, but only 4% on Microsoft virtual world. That is the power of first mover advantage. Given the push towards light-weight federated systems, the economic lock-in of the big four, IBM, Microsoft, Oracle, and SAP is finally broken. Agility is king again. For example, look at this of &lt;a href=&quot;http://www.datamasher.org&quot;&gt;www.datamasher.org&lt;/a&gt;</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/5344525430707798107/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=5344525430707798107' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5344525430707798107'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5344525430707798107'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/12/government-as-platform.html' title='Government as a Platform'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-3903746710914624014</id><published>2009-11-28T10:36:00.000-08:00</published><updated>2009-11-28T10:52:51.682-08:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="silicon innovation"/><category scheme="http://www.blogger.com/atom/ns#" term="venture capital"/><title type='text'>And now something completely different: brain simulation</title><content type='html'>Our good folks at the national labs have been developing cloud computing for two decades, so what they are doing now is possibly an indication of what we&#39;ll be doing with the cloud a decade from now. Researchers at IBM Almaden have been working on the &lt;a href=&quot;http://www.popularmechanics.com/technology/industry/4337190.html&quot;&gt;largest brain simulation&lt;/a&gt;  to date; 1.6 billion neurons with 9 trillion connections. The scale of this endeavor still dwarfs the capacity and capability of any commercial cloud offering; the simulation uses roughly 150 thousand processors and 150TBytes of memory.&lt;br /&gt;&lt;br /&gt;Just to provide a sense of the OPEX of such an installation: &lt;span style=&quot;font-style:italic;&quot;&gt;Dawn, a IBM Blue Gene/P supercomputer at LLNL, hums and breathes inside an acre-size room on the second floor of the lab&#39;s Terascale Simulation Facility. Its 147,456 processors and 147,000 gigabytes of memory fill 10 rows of computer racks, woven together by miles of cable. Dawn devours a million watts of electricity through power cords as thick as a bouncer&#39;s wrists—racking up an annual power bill of $1 million. The roar of refrigeration fans fills the air: 6675 tons of air-conditioning hardware labor to dissipate Dawn&#39;s body heat, blowing 2.7 million cubic feet of chilled air through the room every minute. &lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Given the fact that a real brain only consumes about 25Watts, clearly there is a lot of room for technology innovation. Silicon innovation however has come to a stand still with venture capital completely abandoning this segment. There are no VC firms in the US or EU that have any funds that target this vertical. It is rumored that Google is designing its own silicon now since no commercial chip manufacturers are providing the innovation that Google needs.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/3903746710914624014/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=3903746710914624014' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3903746710914624014'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3903746710914624014'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/11/and-now-something-completely-different.html' title='And now something completely different: brain simulation'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-6222145906993029394</id><published>2009-11-20T10:47:00.001-08:00</published><updated>2009-11-20T11:01:24.371-08:00</updated><title type='text'>Governmental IT: Analytics is not a dirty word</title><content type='html'>Over at Smart Data Collective, &lt;a href=&quot;http://smartdatacollective.com/Home/22568&quot;&gt;Bill Cooper&lt;/a&gt; wrote a wonderful article on deep analytics. In particular, I liked his assessment on the resistance expressed by customers that can&#39;t see the forest for the trees.&lt;br /&gt;&lt;br /&gt;&lt;i&gt;I’ve watched many government agencies balk at the idea of data mining and complex analytics. They are concerned about switching to a new data architecture and the potential risks involved in implementing a new solution or making a change in methodology.&lt;br /&gt;&lt;br /&gt;Having been there, I do understand their concerns, but fear of change is what’s holding government agencies back from being able to fully leverage the data that already exists to effect change at the local, regional, state and national levels. Analytics are the key to lowering costs, increasing revenue and streamlining government programs.&lt;br /&gt;&lt;br /&gt;In my own government experience and now, looking at it from the other side, I have come to believe that government clients need to think about data the way the world’s top corporations do. Like all federal agencies, these companies already had huge repositories of data that were never analyzed – never used to support decisions, plan strategies or take immediate actions. Once they began to treat that data as a corporate asset, they started to see real results. The best part is that leveraging these mountains of data does not require a &quot;rip and replace&quot; approach. Inserting a data warehousing/data mining or complex analytics capability into a SOA or cloud computing environment can be very low risk and even elegant in its implementation. The potential rewards are immense!&lt;br /&gt;&lt;br /&gt;That’s what’s needed in the government sector. We need to view analytics not as a dirty word but as a secret weapon against fraud and other challenges impacting all areas of the government sector.&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;I am a big believer that the future of the cloud consists of federated systems for the simple reason that large data sets are captive to their storage devices. Federated systems makes service oriented architectures (SOA) a natural architecture pattern to collate information. The fact that Google Gears and Microsoft Azure exhibit SOA at different levels of abstraction is clear evidence of the power of SOA. Add coarse grain SOAs to these fine-grained patterns and you can support federation and scale internal IT systems even if the core runs in Gears or Azure.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/6222145906993029394/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=6222145906993029394' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6222145906993029394'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/6222145906993029394'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/11/governmental-it-analytics-is-not-dirty.html' title='Governmental IT: Analytics is not a dirty word'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-4339988303224838318</id><published>2009-11-20T06:25:00.000-08:00</published><updated>2009-11-20T10:22:59.116-08:00</updated><title type='text'>Interactive Map of cloud services</title><content type='html'>Appirio, a company that helps enterprise customers leverage PaaS cloud platforms such as Salesforce.com and Google Apps, put a nice interactive navigator on their website.&lt;br /&gt;The Appirio &lt;a href=&quot;http://www.appirio.com/ecosystem/&quot;&gt;cloud computing ecosystem map&lt;/a&gt; aims to provide more clarity in the fast evolving cloud services market. It tries to help enterprise decision makers to accelerate their adoption of the cloud by trying to provide a standard taxonomy.&lt;br /&gt;&lt;br /&gt;&lt;a onblur=&quot;try {parent.deselectBloggerImageGracefully();} catch(e) {}&quot; href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX_QjmLEngjg5y_n-wiWgkvMB_n7psxBbmC1P6cQYRFQiMOQ-k0NYOxZvpSZF4ZF3msRBEGWRs_5UG0yM5G0FCYvNZjvJTSEYs4Sx1LdYW69VSjfMfL8Q4AgaIBnPAKDfdLFvz_c5R0ZWO/s1600/cloud-eco-system.jpg&quot;&gt;&lt;img style=&quot;display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 343px;&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX_QjmLEngjg5y_n-wiWgkvMB_n7psxBbmC1P6cQYRFQiMOQ-k0NYOxZvpSZF4ZF3msRBEGWRs_5UG0yM5G0FCYvNZjvJTSEYs4Sx1LdYW69VSjfMfL8Q4AgaIBnPAKDfdLFvz_c5R0ZWO/s400/cloud-eco-system.jpg&quot; border=&quot;0&quot; alt=&quot;&quot;id=&quot;BLOGGER_PHOTO_ID_5406196696580068114&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Ryan Nichols, head of cloud strategy at Appirio, states: &lt;span style=&quot;font-style:italic;&quot;&gt;&quot;The cloud ecosystem is evolving so quickly that it&#39;s difficult for most enterprises to keep up. We created the ecosystem map to track this evolution ourselves, and have decided to publish it to help others assess the lay of the land. With broader community involvement, we can create a living, breathing map where anyone can access, drill down and interact with dynamic information. This will bring some much-needed clarity to the cloud market.&quot;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Unfortunately, since the map is geared towards the enterprise customer it ignores all of the innovation that is taking place in the mashup, programmable web, and mid-market products, such as &lt;a href=&quot;http://www.zementis.com/on-the-cloud.htm&quot;&gt;Zementis ADAPA in the Cloud&lt;/a&gt;. Given the new ways in which the cloud enables new application architectures and services, the enterprise market is the worst indicator of the evolving cloud ecosystem.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/4339988303224838318/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=4339988303224838318' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/4339988303224838318'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/4339988303224838318'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/11/interactive-map-of-cloud-services.html' title='Interactive Map of cloud services'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX_QjmLEngjg5y_n-wiWgkvMB_n7psxBbmC1P6cQYRFQiMOQ-k0NYOxZvpSZF4ZF3msRBEGWRs_5UG0yM5G0FCYvNZjvJTSEYs4Sx1LdYW69VSjfMfL8Q4AgaIBnPAKDfdLFvz_c5R0ZWO/s72-c/cloud-eco-system.jpg" height="72" width="72"/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-3559653160056025782</id><published>2009-11-02T11:47:00.000-08:00</published><updated>2009-11-02T11:55:05.125-08:00</updated><title type='text'>PC sales decline</title><content type='html'>In his post &lt;a href=&quot;http://www.cio.com/article/506037/PCs_at_a_Crossroads?source=CIONLE_nlt_digest_2009-11-02&quot;&gt;PCs at a Crossroads&lt;/a&gt; Michael Friedenberg reports on IDC&#39;s measurement of the PC marketplace. From the article:&lt;br /&gt;&lt;br /&gt;&lt;i&gt;&quot;Case in point is the PC market. Market researcher IDC reports that 2009 will be the first year since 2001 where PC shipments will decline. I believe this drop is driven by a more rapid intersection of the cyclical and the systemic as the PC value proposition is challenged and then transformed. As Intel CEO Paul Otellini recently said, &quot;We&#39;re moving from personal computers to personal computing.&quot; That comment signals Intel&#39;s way of moving into new markets, but it also acknowledges that the enterprise PC market has arrived at a crossroads.&quot;&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;Cloud computing is one development that is dramatically changing the desktop and server eco-system. The performance of a single server or desktop hasn&#39;t kept pace with the computational needs of modern science, engineering, or business. Cloud computing moves away from capital equipment to the ability to procure just the computational output AND at infinite scale for most use cases. Most desktops are idling most of the time, but are too slow to get real work done when you need it. This is pushing the work towards elastic resources that are consumed as you go. If the browser is all you need, then a move towards server consolidation and thin clients is not far behind.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/3559653160056025782/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=3559653160056025782' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3559653160056025782'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/3559653160056025782'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/11/pc-sales-decline.html' title='PC sales decline'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-8222530828268353206</id><published>2009-08-27T05:07:00.000-07:00</published><updated>2009-08-27T05:24:27.592-07:00</updated><title type='text'>Amazon Virtual Private Cloud</title><content type='html'>Yesterday, Amazon introduced &lt;a href=&quot;http://aws.typepad.com/aws/2009/08/introducing-amazon-virtual-private-cloud-vpc.html&quot;&gt;Amazon VPC&lt;/a&gt;. It enables logically isolated compute instances and a VPN tunnel to connect to internal data center resources. The architecture is straight forward and Amazon&#39;s blog post depicts is as follows.&lt;br /&gt;&lt;a onblur=&quot;try {parent.deselectBloggerImageGracefully();} catch(e) {}&quot; href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT20siBwkSeNfk924Pcwiacvtv9zAxiEVuHjrqTd-c-pNuaTIV22IrSPoKr5KqZCkldCnpd5OlVU0QaNiCpSNeTOXEcpfwU9CdOm6hzdVdpcZgy0XhX1o6poX7UbVA3wOlqE8oXukcy8HN/s1600-h/amazon-private-cloud.jpg&quot;&gt;&lt;img style=&quot;display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 273px;&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT20siBwkSeNfk924Pcwiacvtv9zAxiEVuHjrqTd-c-pNuaTIV22IrSPoKr5KqZCkldCnpd5OlVU0QaNiCpSNeTOXEcpfwU9CdOm6hzdVdpcZgy0XhX1o6poX7UbVA3wOlqE8oXukcy8HN/s400/amazon-private-cloud.jpg&quot; border=&quot;0&quot; alt=&quot;&quot;id=&quot;BLOGGER_PHOTO_ID_5374615746135838322&quot; /&gt;&lt;/a&gt;&lt;br /&gt;But the implications of VPC are far reaching; there are no real hurdles left to leverage Amazon&#39;s cloud except for limited and costly Internet bandwidth. Amazon&#39;s offering is morphing into a very flexible IaaS with some content delivery network features that are great for geographically dispersed small businesses. I am thinking particularly companies such as Stillwater that are differentiating through high domain expertise. Global talent cannot be bound to a small locale such as Silicon Valley anymore. We are connecting researchers in US, EU, Middle East, and Asia and with offerings like this we can create a development process that follows the moon that rivals the mega-vendor infrastructures. We do not need to uproot any folks to make this happen. These are exciting times we live in that can really unleash the creative spirit of the world.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/8222530828268353206/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=8222530828268353206' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/8222530828268353206'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/8222530828268353206'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/08/amazon-virtual-private-cloud.html' title='Amazon Virtual Private Cloud'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT20siBwkSeNfk924Pcwiacvtv9zAxiEVuHjrqTd-c-pNuaTIV22IrSPoKr5KqZCkldCnpd5OlVU0QaNiCpSNeTOXEcpfwU9CdOm6hzdVdpcZgy0XhX1o6poX7UbVA3wOlqE8oXukcy8HN/s72-c/amazon-private-cloud.jpg" height="72" width="72"/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-527825732316782407</id><published>2009-07-18T12:02:00.000-07:00</published><updated>2009-07-18T12:35:00.536-07:00</updated><title type='text'>Cloud Computing Taxonomy</title><content type='html'>I found this wonderful graphic created by Peter Laird in his &lt;a href=&quot;http://peterlaird.blogspot.com/2009/05/cloud-computing-taxonomy-at-interop-las.html&quot;&gt;blog&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;&lt;a onblur=&quot;try {parent.deselectBloggerImageGracefully();} catch(e) {}&quot; href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2S3XPru1yq9J9zgE6y0OehM9CRFGH0tY3bc4x1_nMr_i398J3hL4B6UsrCg4jfnVlB9GCILUS9Ee_IW_sdngz2YLTca5Vrn5p3-CltS0n8QNdP8jvsTuomyY7GAq7sgVp_Oe_HMwi01H1/s1600-h/saas-cloud-universe.png&quot;&gt;&lt;img style=&quot;display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 336px; height: 400px;&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2S3XPru1yq9J9zgE6y0OehM9CRFGH0tY3bc4x1_nMr_i398J3hL4B6UsrCg4jfnVlB9GCILUS9Ee_IW_sdngz2YLTca5Vrn5p3-CltS0n8QNdP8jvsTuomyY7GAq7sgVp_Oe_HMwi01H1/s400/saas-cloud-universe.png&quot; border=&quot;0&quot; alt=&quot;&quot;id=&quot;BLOGGER_PHOTO_ID_5359878365536492114&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Peter&#39;s &lt;a href=&quot;http://peterlaird.blogspot.com/2009/05/cloud-computing-taxonomy-at-interop-las.html&quot;&gt;blog&lt;/a&gt; has all the descriptions of the buckets.&lt;br /&gt;&lt;br /&gt;The Public Cloud bucket is heavily underreported. There are roughly about 1200 public data centers in the US alone that are quite happy to rent you a server or cabinet. There are a host of data center market places that will connect you to a data center provider. Here are a few:&lt;br /&gt;&lt;br /&gt;&lt;a href=&quot;http://www.findadatacenter.com/&quot;&gt;Find a Data Center&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;&lt;a href=&quot;http://www.datacenterknowledge.com&quot;&gt;Data Center Knowledge&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;&lt;a href=&quot;http://www.datacentermarketplace.com/&quot;&gt;Data Center Marketplace&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;In particular, the telecom companies, like 365 Main, SuperNAP, Qwest, Verizon, Level 3 Communications are quite happy to sell you connectivity AND servers and are perfect for large cloud deployments that need geographic spread and high bandwidth.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/527825732316782407/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=527825732316782407' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/527825732316782407'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/527825732316782407'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/07/cloud-computing-taxonomy.html' title='Cloud Computing Taxonomy'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2S3XPru1yq9J9zgE6y0OehM9CRFGH0tY3bc4x1_nMr_i398J3hL4B6UsrCg4jfnVlB9GCILUS9Ee_IW_sdngz2YLTca5Vrn5p3-CltS0n8QNdP8jvsTuomyY7GAq7sgVp_Oe_HMwi01H1/s72-c/saas-cloud-universe.png" height="72" width="72"/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-2311713362615784280</id><published>2009-07-14T08:11:00.000-07:00</published><updated>2009-07-14T08:24:27.917-07:00</updated><title type='text'>On-demand pricing for Windows Azure</title><content type='html'>InformationWeek&#39;s Paul McDougall &lt;a href=&quot;http://www.informationweek.com/news/windows/operatingsystems/showArticle.jhtml?articleID=218500341&quot;&gt;reports&lt;/a&gt; on Windows Azure pricing and it provides confirmation that Microsoft is transitioning its boxed software business into a service business. &lt;br /&gt;&lt;br /&gt;Paul&#39;s assessment: &lt;br /&gt;&lt;span style=&quot;font-style:italic;&quot;&gt;&quot;Azure is the latest sign that Microsoft is eyeing the Web as the primary delivery mechanism for software and services. On Monday, the company said it planned to make a version of Microsoft Office 2010 available to consumers over the Internet at no charge. It plans a similar offering for businesses.&quot;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;In my mind there is still one piece missing for productive cloud computing and that is the seamless integration of the client. None of the big vendors are particularly keen on solving this problem since it diminishes their economic lock-in. But users create, use, and transform data and information on their clients and data needs to seamlessly flow between the client and the cloud. This flow in my mind is best managed by the OS, or a tightly integrated run-time. You see many of these service components show up in the mobile platforms, but the PC ecosystem is lagging here.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/2311713362615784280/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=2311713362615784280' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/2311713362615784280'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/2311713362615784280'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/07/on-demand-pricing-for-windows-azure.html' title='On-demand pricing for Windows Azure'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-1567559963493644732</id><published>2009-07-14T05:54:00.000-07:00</published><updated>2009-07-14T06:31:26.398-07:00</updated><title type='text'>Netbooks and the cloud</title><content type='html'>Dana Blankenhorn at ZDNet posted an interesting &lt;a href=&quot;http://blogs.zdnet.com/open-source/?p=4493&quot;&gt;analysis&lt;/a&gt; of Google&#39;s Chrome OS announcement. The basic premise is that Google as a cloud information provider can subsidize a Netbook since it will get it back in cloud service revenue and a higher intangible value to its core business of collecting and characterizing customer behavior. &lt;br /&gt;&lt;br /&gt;This is much like the telecom business or the game console business, and I have heard that same story from the reps at Samsung, Nokia, Asus, and Sony. It is just that Google has a big head start in the intangible value department. &lt;br /&gt;&lt;br /&gt;But buried in this article is the core observation in my mind why the boxed world of software is transitioning to the cloud: security and cost.&lt;br /&gt;&lt;br /&gt;&lt;span style=&quot;font-style:italic;&quot;&gt;&quot;The problem is that Netbooks are cheap and, while they will gain in power they will stay cheap. I spent $270 on my HP Mini and that’s about right.&lt;br /&gt;&lt;br /&gt;Microsoft has reportedly cut the price of Windows to $3 to capture Netbook OEMs, and it’s offering a cut-rate price on Office, too.&lt;br /&gt;&lt;br /&gt;But when you consider the $50/year price to license an anti-viral, the $30/year to license a malware program and the additional $30/year you need for a registry cleaner, the software price of a Netbook gets completely out of line with its hardware cost.&quot;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;This is the same observation that can be used for any boxed software. The cost of the underlying hardware platform has shrunk in the past 20 years, but the software cost hasn&#39;t kept pace. 20 years ago a workstation cost $75k so a $75k piece of software was reasonable. The cost of a workstation is now $2k, but the software is still $75k. The productivity improvement that I need to get from the software to justify the cost is too high and thus that type of cost can only be carried by a business model that has significant intangible value. And that value isn&#39;t present in the consumer and/or SMB market. &lt;br /&gt;&lt;br /&gt;The smart phone started this trend and the netbook will accelerate it: the bulk of the market will be delivered services through subsidized hardware and software and it is the service providers that call the shots. Google, Amazon, Microsoft, Apple, Sony are already transitioning into these roles and since they have a connection with the bottom of the market pyramid, they will attract so much money that they will quickly roll over the Adobes, Oracles and SAPs of the world. &lt;br /&gt;&lt;br /&gt;Many independent software vendors will clamor on the infrastructures of Google, Amazon, and Apple, and intangible value will be created. The enterprise market, of all markets, can&#39;t be isolated from the bulk of the money and they will need to adapt to the system where the information resides: and that will be the cloud.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/1567559963493644732/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=1567559963493644732' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/1567559963493644732'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/1567559963493644732'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/07/netbooks-and-cloud.html' title='Netbooks and the cloud'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-5135839081679273496</id><published>2009-06-15T09:30:00.000-07:00</published><updated>2009-06-15T10:10:31.518-07:00</updated><title type='text'>Eight ways that cloud computing will change your business</title><content type='html'>&lt;a href=&quot;http://blogs.zdnet.com/Hinchcliffe/?p=488&quot;&gt;Eight Ways that Cloud Computing Will Change Business&lt;/a&gt; is a wonderful post by Dion Hinchcliffe. The synopsis of this article is that large businesses are laggards with respect to technology adoption for the simple reason that the cost of betting on the wrong horse is too high. However, sometimes new technologies are so compelling that this wait-and-see approach is trumped. According to the article:&lt;br /&gt;&lt;br /&gt;&lt;i&gt;&quot;Cloud Computing is quickly beginning to shape up as one of these major changes and the hundreds of thousands of business customers of cloud offerings from Amazon, Salesforce, and Google, including a growing number of Fortune 500 companies, is showing both considerable interest and momentum in the space&quot;.&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;The article continues to spell out eight ways cloud computing will change business.&lt;br /&gt; &lt;br /&gt;&lt;nl&gt;&lt;li&gt;Creation of a new generation of products and services&lt;/li&gt;&lt;li&gt;New lightweight form of real-time partnerships and outsourcing with IT suppliers&lt;/li&gt;&lt;li&gt;New awareness and leverage of the greater Internet and Web 2.0 in particular&lt;/li&gt;&lt;li&gt;A reconciliation of traditional SOA with cloud and other emerging IT models&lt;/li&gt;&lt;li&gt;The rise of new industry leaders and IT vendors&lt;/li&gt;&lt;li&gt;More self-service IT from the business-side&lt;/li&gt;&lt;li&gt;More tolerance for innovation and experimentation from business&lt;/li&gt;&lt;li&gt;The slow-moving, dinosaur firms will have trouble keeping up with more nimble adopters and fast-followers&lt;/li&gt;&lt;/nl&gt;&lt;br /&gt;&lt;br /&gt;I have always argued that cloud computing will be defined by the bottom of the economic pyramid. Smaller businesses do not have existing and legalized corporate standards of quality, accountability, and security, and they can simply piggyback on the standards provided by the data centers on which they deploy. This provides them with a first mover advantage that doesn&#39;t waste energy trying to sell cloud computing solutions inside an already stressed IT organization of a large enterprise. &lt;br /&gt;&lt;br /&gt;Secondly, consumers in many ways are much more adaptable than enterprises. I am using Google or Amazon or AT&amp;T but I don&#39;t get bend out of shape if my service experiences a hick-up. Take cell phone service: if you insisted on 99.999% availability, like many enterprise customers seem to demand, you couldn&#39;t use a cell phone. However, everybody agrees that a cell phone is a net productivity improvement. It is this consumer, conditioned by an imperfect world, that is demanding new services for their iPhones, BlackBerries, and Pres and is willing to take a less stringent SLA in exchange for lower cost and convenience. And there is a legion of startups that is willing to test out that appetite. &lt;br /&gt;&lt;br /&gt;Brand loyalty in this connected world is non-existent for the simple reason that most services are multi-vendor anyways. You get a Nokia phone on a Verizon network connecting to a Real Rhapsody music service to satisfy your need for mobility. I switched from Yahoo search, to Google search, to Microsoft search in a matter of minutes simply because either their UI and/or their results provided a better fit for my sensibilities. I find it wonderful that after a decade of technology consolidation and stagnation we are back to a world of innovation and rapid expansion of new services. And I believe that it is the consumer that will define these services, not the enterprise.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/5135839081679273496/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=5135839081679273496' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5135839081679273496'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/5135839081679273496'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/06/eight-ways-that-cloud-computing-will.html' title='Eight ways that cloud computing will change your business'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-715725806728995718.post-7019644450943423475</id><published>2009-05-15T07:01:00.000-07:00</published><updated>2009-05-15T09:32:09.186-07:00</updated><title type='text'>Amazon EC-2 for Compute Intensive Workloads</title><content type='html'>The cloud has evolved from the managed hosting concept. With data centers like EC-2 making it easier to provision servers on-demand, elasticity can be build into the application to scale dynamically. Microsoft Azure provides a similar, and nicely integrated, platform for the Windows application world. But how well do this clouds hold up when demand is elastic for compute intensive workloads? The short of it? Not so well.&lt;br /&gt;&lt;br /&gt;I found two papers that report on experiments that take Amazon EC-2 as IT fabric and deploy compute intensive workloads on them. They compare these results to the performance obtained from on-premise clusters that include best-known practices for compute intensive workloads. The first &lt;a href=&quot;http://www.usenix.org/publications/login/2008-10/openpdfs/walker.pdf&quot;&gt;paper&lt;/a&gt; uses the NAS benchmarks to get a broad sampling of high-performance computing workloads on the EC-2 cloud. They use the high-performance instances of Amazon and compare them to similar processor gear in a cluster at NCSA. The IT gear details are shown in the following table:&lt;br /&gt;&lt;br /&gt;&lt;table border=&quot;2&quot;&gt;&lt;tr&gt;&lt;th&gt;&lt;/th&gt;&lt;th&gt;EC-2 High-CPU Cluster&lt;/th&gt;&lt;th&gt;NCSA Cluster&lt;/th&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Compute Node&lt;/td&gt;&lt;td&gt;7GB memory, 4 cores per socket, 2 sockets per server, 2.33GHz Xeon, 1600GB storage&lt;/td&gt;&lt;td&gt;8GB memory, 4 cores per socket, 2 sockets per server, 2.33GHz Xeon, 73GB storage&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Network Interconnect&lt;/td&gt;&lt;td&gt;Specific Interconnect technology unknown&lt;/td&gt;&lt;td&gt;Infiniband network&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;br /&gt;&lt;br /&gt;The NAS Parallel Benchmarks are a widely used set of programs designed to evaluate the performance of high performance computing systems. The suite mimics critical computation and data movement patterns important for compute intensive workloads. &lt;br /&gt;&lt;br /&gt;Clearly, when the workload is confined to a single server the difference between the two compute environments is limited to the virtualization technology used and effective available memory. In this experiment the difference between Amazon EC-2 and a best-known practice cluster is between 10-20% in favor of a non-virtualized server. &lt;br /&gt;&lt;br /&gt;However, when the workload needs to reach across the network to other servers to complete its task the performance difference is striking, as is shown in the following figure.&lt;br /&gt;&lt;br /&gt;&lt;a onblur=&quot;try {parent.deselectBloggerImageGracefully();} catch(e) {}&quot; href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoa04rG2G69l3vC0knEJj_INhtaGPOMEV1NmNtQaHE-uR-ydvj1EkeQ0pVjjTRT1HaqsZGkYvFJrFdTCItdFbfiU1RWAPcZnM03LH6Rpf_dI5ni0ZlnNKOFP-U4-ObuImHkp7QJL1LNHRZ/s1600-h/cluster-performance-amazon-ec2.bmp&quot;&gt;&lt;img style=&quot;display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 260px;&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoa04rG2G69l3vC0knEJj_INhtaGPOMEV1NmNtQaHE-uR-ydvj1EkeQ0pVjjTRT1HaqsZGkYvFJrFdTCItdFbfiU1RWAPcZnM03LH6Rpf_dI5ni0ZlnNKOFP-U4-ObuImHkp7QJL1LNHRZ/s400/cluster-performance-amazon-ec2.bmp&quot; border=&quot;0&quot; alt=&quot;&quot;id=&quot;BLOGGER_PHOTO_ID_5336057407144417954&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Figure 1: NPB-MPI runtimes on 32 cores (= 4 dual socketed servers)&lt;br /&gt;&lt;br /&gt;The performance difference ranges from 2x to 10x in favor of a optimized high-performance cluster. Effectively, Amazon is ten times more expensive to operate than if you had your own optimized equipment. &lt;br /&gt;&lt;br /&gt;The second &lt;a href=&quot;http://www.cs.utexas.edu/users/pauldj/pubs/uchpc09.pdf&quot;&gt;paper &lt;/a&gt;talks to the cost adder of using cloud computing IT infrastructure for compute intensive workloads. In this experiment, they use a common workload to measure the performance of a supercomputer, HPL, which is an acronym for High Performance LINPACK. HPL is relatively kind to a cluster in the sense that it does not tax the interconnect bandwidth/latency much as compared to other compute intensive workloads such as optimization, information retrieval, or web indexing. The experiment measures the average floating point operations (FLOPS) obtained divided by the average compute time used. This experiments shows an exponential decrease in performance with respect to dollar cost of the clusters. This implies that if we double the cluster size the FLOPS/sec for money spent does down.&lt;br /&gt;&lt;br /&gt;The first paper has a wonderful graph that explains what is causing this weak scaling result. &lt;br /&gt;&lt;a onblur=&quot;try {parent.deselectBloggerImageGracefully();} catch(e) {}&quot; href=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhx3zwcvXvMdEeM7VBu89urWmEGi3m0F22Z9rByP54dA2MjK1C9PCdEPcqXJ4ohf7EizwYuVxEc-b7jd-K5IJqvFGe52iHrWs9N0ZM31USk1BXaAxFD9ZF5rdBhtdSZ38xVtiUtYPOEHcGh/s1600-h/bisection-bw-amazon-ec2.bmp&quot;&gt;&lt;img style=&quot;display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 248px;&quot; src=&quot;https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhx3zwcvXvMdEeM7VBu89urWmEGi3m0F22Z9rByP54dA2MjK1C9PCdEPcqXJ4ohf7EizwYuVxEc-b7jd-K5IJqvFGe52iHrWs9N0ZM31USk1BXaAxFD9ZF5rdBhtdSZ38xVtiUtYPOEHcGh/s400/bisection-bw-amazon-ec2.bmp&quot; border=&quot;0&quot; alt=&quot;&quot;id=&quot;BLOGGER_PHOTO_ID_5336063740301252818&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;This figure shows the bisection bandwidth of the Amazon EC-2 cluster and that of a best-known practice HPC cluster. Bisection bandwidth is the bandwidth between two equal parts of a cluster. It is a measure how well-connected all the servers are to one another. The focus of typical clouds to provide a productive and high margin service pushes them into IT architectures that do not favor interconnect bandwidth between servers. Many clouds are commodity servers connected to a SAN and the bandwidth is allocated to that path, not to bandwidth between servers. And that is opposite to what high performance clusters for compute intensive workloads have evolved to. &lt;br /&gt;&lt;br /&gt;This means that for the enterprise class problems, were efficiency of IT equipment is a differentiator to solve the problems at hand, cloud IT infrastructure solutions are not well matched yet. However, for SMBs that are seeking mostly elasticity and on-demand use, cloud solutions still work since there are still monetary benefits to be extracted from deploying compute intensive workloads on Amazon or other clouds.</content><link rel='replies' type='application/atom+xml' href='http://stillwater-cse.blogspot.com/feeds/7019644450943423475/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=715725806728995718&amp;postID=7019644450943423475' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/7019644450943423475'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/715725806728995718/posts/default/7019644450943423475'/><link rel='alternate' type='text/html' href='http://stillwater-cse.blogspot.com/2009/05/amazon-ec-2-for-compute-intensive.html' title='Amazon EC-2 for Compute Intensive Workloads'/><author><name>Theodore Omtzigt</name><uri>http://www.blogger.com/profile/18249997607495408266</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='22' height='32' src='//blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi0rt2tCgV-qT7qCs0Dg79Xj_93raMz1JwXekpW-EPJq4IQIbnrKlYzM3enpe1abIloOH-HBq0nZQYb0MXAfJeoeA35FCkJHCIo5lIcGoKfBTb5OCJKyJNLN_q5V8TIQ/s220/2011-12-13-digital-signature.jpg'/></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoa04rG2G69l3vC0knEJj_INhtaGPOMEV1NmNtQaHE-uR-ydvj1EkeQ0pVjjTRT1HaqsZGkYvFJrFdTCItdFbfiU1RWAPcZnM03LH6Rpf_dI5ni0ZlnNKOFP-U4-ObuImHkp7QJL1LNHRZ/s72-c/cluster-performance-amazon-ec2.bmp" height="72" width="72"/><thr:total>2</thr:total></entry></feed>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

  1. Download the "valid Atom 1.0" banner.

  2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

  3. Add this HTML to your page (change the image src attribute if necessary):

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//stillwater-cse.blogspot.com/feeds/posts/default

Home · About · News · Docs · Terms

 
Copyright © 2002-9 Sam Ruby, Mark Pilgrim, Joseph Walton, and Phil Ringnalda