tag:blogger.com,1999:blog-53614618346370960192024-03-09T02:59:51.314-08:00Portal StackAnonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.comBlogger42125tag:blogger.com,1999:blog-5361461834637096019.post-26172721337937022972015-01-22T18:41:00.002-08:002015-01-22T18:41:23.502-08:00CICD and Continuous Communication (CICDCC)This is a presentation for test driven infrastructure utilizing continuous integration and delivery tools and realtime communication with slack.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">madisonhub/mean-skel</span><br />
<br />
<br />
Just like most CICD systems, I can push code and have it build it. Whats cool with slack is that I can get notifications, in realtime, from GitLab and GitLab-CI.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7Gp1CPfnnByQQZ9HeGymNSdmGCyXclqo8TlKSdz6-utM6T0iEwFil0Q6PK7vwDbtCp-m3QNtJbPmGF3l9Z7rcewO-DGju2QsyNLIeE62c8T-GXkHQ7Q_Q4znTJ4OQODoHgxeCdV5SMKQ/s1600/Screen+Shot+2015-01-22+at+8.37.47+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7Gp1CPfnnByQQZ9HeGymNSdmGCyXclqo8TlKSdz6-utM6T0iEwFil0Q6PK7vwDbtCp-m3QNtJbPmGF3l9Z7rcewO-DGju2QsyNLIeE62c8T-GXkHQ7Q_Q4znTJ4OQODoHgxeCdV5SMKQ/s1600/Screen+Shot+2015-01-22+at+8.37.47+PM.png" height="156" width="640" /></a></div>
<br />
<br />
The gitlab links will take you directly to the project and commit diffs. Clicking on the gitlab-ci links will allow you to watch build output, in realtime, as well as build charts and graphs.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr17mdIm_gpNRFb1OoskH9rFS-KHm_Yg05XalX3gNuXVrKV6FEq7ERVXQ71aN3Vlm3ct8AUgwZ_5aK37nHxCr_u4QjyyQGWgih9ufLtQAZjJUMvZU6qGJ5XRiT-k2m-VFc5lUrVrNHZ-w/s1600/Screen+Shot+2015-01-22+at+8.04.02+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr17mdIm_gpNRFb1OoskH9rFS-KHm_Yg05XalX3gNuXVrKV6FEq7ERVXQ71aN3Vlm3ct8AUgwZ_5aK37nHxCr_u4QjyyQGWgih9ufLtQAZjJUMvZU6qGJ5XRiT-k2m-VFc5lUrVrNHZ-w/s1600/Screen+Shot+2015-01-22+at+8.04.02+PM.png" height="640" width="542" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdKm6nIYQr1SJENrXJWySWppJeE7tRi_st_XQGIKocdHt23annu4-5KEjFuFWzqpjyArSAXCU2bDyV2H8rakbAZFjPdVb2rrEP9XSXpB3TkLte_5S5f9SAPTcBQH2SxTdTIqUVA4uxf1M/s1600/Screen+Shot+2015-01-22+at+8.05.29+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdKm6nIYQr1SJENrXJWySWppJeE7tRi_st_XQGIKocdHt23annu4-5KEjFuFWzqpjyArSAXCU2bDyV2H8rakbAZFjPdVb2rrEP9XSXpB3TkLte_5S5f9SAPTcBQH2SxTdTIqUVA4uxf1M/s1600/Screen+Shot+2015-01-22+at+8.05.29+PM.png" height="640" width="482" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Since gitlab-ci shows the build is passing, its likely the build will work if done in a production container. Ill create a container and add it to my salt-master environment.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"># docker run -d --name mean-skel -h mean-skel.docker01 -t docker-minion</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"># docker inspect mean-skel | grep IPAddress</span><br />
<span style="font-family: Courier New, Courier, monospace;"> "IPAddress": "172.17.0.14",</span><br />
<br />
Now that I know the IP address of the container, I need to add it to my servers pillar and tell nagios to reload.<br />
<br />
<b>/srv/pillar/servers.sls:</b><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> mean-skel:</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ip: 172.17.0.14</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> node_type: container</span><br />
<span style="font-family: Courier New, Courier, monospace;"> nagioscfg:</span><br />
<span style="font-family: Courier New, Courier, monospace;"> use: linux-server</span><br />
<span style="font-family: Courier New, Courier, monospace;"> parents: docker01</span><br />
<br />
<b>salt 'nagios*' state.sls nagios.running container</b><br />
<br />
I am now monitoring it, in realtime.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYeHMGCtIhxSLeEaE0Y1FFy2icXRjdkvwPWjdOA8JMCMLIRIj1CfFMYr7oPwv-Vfy0-RoaBiVj66tDz0Iopj3PyWcVwUYu0zaTeKcNrNgB4iGxr-fGy0p7XocQV7_Pkyc4WzwhLRvqHns/s1600/Screen+Shot+2015-01-22+at+8.16.09+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYeHMGCtIhxSLeEaE0Y1FFy2icXRjdkvwPWjdOA8JMCMLIRIj1CfFMYr7oPwv-Vfy0-RoaBiVj66tDz0Iopj3PyWcVwUYu0zaTeKcNrNgB4iGxr-fGy0p7XocQV7_Pkyc4WzwhLRvqHns/s1600/Screen+Shot+2015-01-22+at+8.16.09+PM.png" height="522" width="640" /></a></div>
<br />
<br />
By default, I monitor for salt-minion process and users. Since the new container isn't in its final highstate, I am getting notifications.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjy123ozfMjjBJourKOCi0SHzSEvUx1-E8z8oCrdpxIJJqIy1z5XMGlNlBH2fNMPIfnwW52r-SP3A-_0PTDpgxrlXFmUBTXMuKfhRcw2UCYaDaHPls-HE-n_3GACWK26ydPRa_0m4mT4Go/s1600/Screen+Shot+2015-01-22+at+8.17.33+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjy123ozfMjjBJourKOCi0SHzSEvUx1-E8z8oCrdpxIJJqIy1z5XMGlNlBH2fNMPIfnwW52r-SP3A-_0PTDpgxrlXFmUBTXMuKfhRcw2UCYaDaHPls-HE-n_3GACWK26ydPRa_0m4mT4Go/s1600/Screen+Shot+2015-01-22+at+8.17.33+PM.png" height="175" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Since I decided not to automatically install the nrpe agent, I am getting those errors. Notice that "CRITICAL" is highlighted yellow. That is a custom filter I am looking for specifically in the alert channel on slack. If I am not actively in slack, I will get notified on my phone with the slack app of critical events in my infrastructure.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
salt 'mean-skel*' state.sls nagios.nrpe container</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6sUMDnjxRZK0hJxSWdoGowRChNQ7GxXnWR0xZ-vQh7RiZ6AK4MeuhX3eb12wWCnXxR6QztkZLWebEzcUA4haQMJs4jxb9BWTv1aEIp_7yks1zekEQvAJXLNVIstxYfCE140QuS7qxoi4/s1600/Screen+Shot+2015-01-22+at+8.25.09+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6sUMDnjxRZK0hJxSWdoGowRChNQ7GxXnWR0xZ-vQh7RiZ6AK4MeuhX3eb12wWCnXxR6QztkZLWebEzcUA4haQMJs4jxb9BWTv1aEIp_7yks1zekEQvAJXLNVIstxYfCE140QuS7qxoi4/s1600/Screen+Shot+2015-01-22+at+8.25.09+PM.png" height="150" width="400" /></a><br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Slack has notified me that services have come back. Taking a look at the Tactical Overview in nagios, I can see that everything is OK.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFpMdAIYc7eqKI3oSMLqsX1bQoOxQkqq2Gauu_5RWbNUcLoSunGAhkBv6BgAK5tOQUe2j0IXlF_MF1-cZ8O5jNCs4WsF2UZZgpgb3rPL3TTjrYzEhWcDnZTM3kFm3K-51zljSwILg-SnI/s1600/Screen+Shot+2015-01-22+at+8.24.57+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFpMdAIYc7eqKI3oSMLqsX1bQoOxQkqq2Gauu_5RWbNUcLoSunGAhkBv6BgAK5tOQUe2j0IXlF_MF1-cZ8O5jNCs4WsF2UZZgpgb3rPL3TTjrYzEhWcDnZTM3kFm3K-51zljSwILg-SnI/s1600/Screen+Shot+2015-01-22+at+8.24.57+PM.png" height="273" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
Whats next? What isn't shown here is the salt-master interface and its hooks into slack for the state of minions. That is next. Stay tuned.</div>
Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com1tag:blogger.com,1999:blog-5361461834637096019.post-13401989020982715852014-09-11T20:33:00.001-07:002014-09-11T20:33:56.490-07:00vagrant-gitlabThe project <a href="https://github.com/MadisonHub/vagrant-gitlab" target="_blank">vagrant-gitlab</a> is a self serve <a href="https://about.gitlab.com/" target="_blank">gitlab</a> install for your local machine. It uses a combination of open source technologies to easily deploy in seconds.<br />
<br />
First, clone the project:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">git clone https://github.com/MadisonHub/vagrant-gitlab.git</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
Then start the virtual machine:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">vagrant up gitlab --provision</span><br />
<br />
<span style="font-family: inherit;">At this point, saltstack will be bootstrapped. Ignore any errors about salt-minion trying to start. We will call salt locally inside the virtual machine.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">vagrant ssh gitlab</span><br />
<span style="font-family: Courier New, Courier, monospace;">sudo salt-call -l debug --local state.highstate</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiziTqkid9wuXmXzTAaTo-585Jzus664zUUdcKlK-Laxfa-tmbBaWCbTEXAnLwFnnfheSN2rknFD92SycefSSYhOT4LnvUre5G0fxScyHH1GBWl-eaFuwYT0qUQTjQj6eY7oM7oqe33l54/s1600/setpasswordgitlab.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiziTqkid9wuXmXzTAaTo-585Jzus664zUUdcKlK-Laxfa-tmbBaWCbTEXAnLwFnnfheSN2rknFD92SycefSSYhOT4LnvUre5G0fxScyHH1GBWl-eaFuwYT0qUQTjQj6eY7oM7oqe33l54/s1600/setpasswordgitlab.jpg" height="200" width="400" /></a>Gitlab should be installed and running on <a href="http://localhost:8888/">http://localhost:8888</a>. Use the username <b><span style="font-family: Courier New, Courier, monospace;">root</span></b> and the password <span style="font-family: Courier New, Courier, monospace; font-weight: bold;">5iveL!fe. </span><span style="font-family: inherit;">You will be prompted to change your password.</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL8H0Yg4IfyeMWukoBmZA6C141y7yhv3FRATPdOvCq79pFgPYzIqgrFoqYtp1ze1DuH2YXr4FG6qhRp_jf4nuNfm-dT4V5sScMEs8rFtarxB9eu6GnAB2Q6HSSL0YE9WwbRaoFD3GKt8o/s1600/gitlab-welcome.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL8H0Yg4IfyeMWukoBmZA6C141y7yhv3FRATPdOvCq79pFgPYzIqgrFoqYtp1ze1DuH2YXr4FG6qhRp_jf4nuNfm-dT4V5sScMEs8rFtarxB9eu6GnAB2Q6HSSL0YE9WwbRaoFD3GKt8o/s1600/gitlab-welcome.jpg" height="225" width="400" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL8H0Yg4IfyeMWukoBmZA6C141y7yhv3FRATPdOvCq79pFgPYzIqgrFoqYtp1ze1DuH2YXr4FG6qhRp_jf4nuNfm-dT4V5sScMEs8rFtarxB9eu6GnAB2Q6HSSL0YE9WwbRaoFD3GKt8o/s1600/gitlab-welcome.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL8H0Yg4IfyeMWukoBmZA6C141y7yhv3FRATPdOvCq79pFgPYzIqgrFoqYtp1ze1DuH2YXr4FG6qhRp_jf4nuNfm-dT4V5sScMEs8rFtarxB9eu6GnAB2Q6HSSL0YE9WwbRaoFD3GKt8o/s1600/gitlab-welcome.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a></div>
You can now create groups and projects. <br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2gs8dr_MtJ1YbD034IgNIuvq0lkZItB6qCv-okDp_s3shflfXUx89nsV-lyCM2EXuB9AX0GPpAkcOVXYUnaaTKf6UW-8AqjBr3x22KuEiI339bja76mCTiJsad-htviXy63s4xH9xxLE/s1600/gitlab-wiki.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2gs8dr_MtJ1YbD034IgNIuvq0lkZItB6qCv-okDp_s3shflfXUx89nsV-lyCM2EXuB9AX0GPpAkcOVXYUnaaTKf6UW-8AqjBr3x22KuEiI339bja76mCTiJsad-htviXy63s4xH9xxLE/s1600/gitlab-wiki.jpg" height="387" width="400" /></a></div>
Each project has a wiki. You can even publish to the wiki using git.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi86CfpqxX6-YyDEcO8kMSQu71u73cLXzvzf5URlTloE3sNU9aY6EnbuUnIf_S2dYZPgJy1Mh9oTri5we9bqgdWVub6GE6J614ne_1f64-gZr-2XcKy0ANcgt9AvITFdGg52WuUr8TRNbg/s1600/gitlab-issue.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi86CfpqxX6-YyDEcO8kMSQu71u73cLXzvzf5URlTloE3sNU9aY6EnbuUnIf_S2dYZPgJy1Mh9oTri5we9bqgdWVub6GE6J614ne_1f64-gZr-2XcKy0ANcgt9AvITFdGg52WuUr8TRNbg/s1600/gitlab-issue.jpg" height="373" width="400" /></a><br />
Each project also has project management with milestones and issues. <br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiypqzdtFpZs3y01EJjG1EY4ANX-yrqFlz5TMmJELHkQB7-mNRQ5WL8tv9FKXNAvmyH5Jk9c4xXpUtT4aXhgag1f5_8JyCMZw-bxb1k5nxEIT0UTlIXeMPtJsx9zO3HjVgOHflXlAjEX8M/s1600/gitlab-milestones.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiypqzdtFpZs3y01EJjG1EY4ANX-yrqFlz5TMmJELHkQB7-mNRQ5WL8tv9FKXNAvmyH5Jk9c4xXpUtT4aXhgag1f5_8JyCMZw-bxb1k5nxEIT0UTlIXeMPtJsx9zO3HjVgOHflXlAjEX8M/s1600/gitlab-milestones.jpg" height="392" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
For backup and rake tasks, head to the github repo and look at the readme.md. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://github.com/MadisonHub/vagrant-gitlab">https://github.com/MadisonHub/vagrant-gitlab</a></div>
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com1tag:blogger.com,1999:blog-5361461834637096019.post-30655599422222912872014-08-02T07:05:00.000-07:002014-12-19T15:26:36.350-08:00Building my own CI/CD Platform<div class="wiki-holder" style="background-color: white; box-sizing: border-box;">
<div class="wiki" style="box-sizing: border-box;">
<h2 style="box-sizing: border-box; color: inherit; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 1.5em; font-weight: 500; line-height: 1.1; margin-bottom: 9px; margin-top: 35px; position: relative;">
Building my own personal CI/CD Platform</h2>
<div style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
One of the more interesting concepts I have been focusing on lately is <a href="http://en.wikipedia.org/wiki/Continuous_integration" target="_blank">continuous integration</a> and development [and (re)deployment] of web based solutions and services. Combined with agile methodologies, social networking, and test driven development, it promotes healthy techniques and strategies to iterate successful solutions based on code for teams of developers and sysadmins alike.<br />
<br />
In this post I will discuss my journey from Vagrant and SaltStack to my new adventure into Mesos. Ill quickly shame on the service discovery movement, and then close up with my progress and current project path researching <a href="http://mesosphere.io/" target="_blank">Mesosphere</a> and <a href="http://en.wikipedia.org/wiki/Software_defined_networking" target="_blank">Software Defined Networking</a>.<br />
<br /></div>
<h3 style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
<span style="font-size: 14px; line-height: 1.6;">Agile Methodologies</span></h3>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGjDgW2LNvz08sWptQC6nDlBFE3yI8GkxU_Kqg9Y6H520le3SGYD6pArGJAmww8tldSkF78GSSPQ_wjVxm6KYcLTAD-4a0p2qyX3eK78Php_xq1Hw_N6eCpsz7hgrh8vBOAtgbxrEfLbM/s1600/Agile-Development.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGjDgW2LNvz08sWptQC6nDlBFE3yI8GkxU_Kqg9Y6H520le3SGYD6pArGJAmww8tldSkF78GSSPQ_wjVxm6KYcLTAD-4a0p2qyX3eK78Php_xq1Hw_N6eCpsz7hgrh8vBOAtgbxrEfLbM/s1600/Agile-Development.png" height="176" width="200" /></a>A common theme in my posts and work ethic includes the concept of <a href="http://en.wikipedia.org/wiki/Agile_software_development" target="_blank">agile software development</a>. It's a modern way to manage a technical project with many moving parts. Breaking down goals and milestones into iterative code changes and pushing them as merge requests is extremely self documenting and keeps others informed and engaged.<br />
<br />
<b>Agile Test Driven Infrastructure</b><br />
Interestingly, this agile software development strategy can be applied to infrastructure as well. With Salt recipes, you can fully provision and manage infrastructure and network devices through a unified control interface. These recipes, which consist of yaml code, can be added to a git workflow. When recipes are changed, gitlab can hook into the commit and merge events and run test cases against the infrastructure with the new recipes. Since Salt recipes consist of states, they will either pass, change, or fail. <br />
<br />
Since we can test the state of infrastructure against code changes, we can treat infrastructure like software and attack the project management of infrastructure with scrum and agile test driven development (but lets call it <i>agile</i> <i>test driven infrastructure</i>).<br />
<br />
SaltStack standardizes the way you manage infrastructure. Because of this, its possible to gauge the time effort necessary for completing a task. For scrum to be effective, tasks need to be scored on a time effort scale (1, 2, 4 hours for example) and assigned appropriately for maximum parallel tasking between team members to fill up the total time effort allotment. <br />
<br /></div>
<h3 style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
</h3>
<h3 style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUNQBuFhLd4QKSXXJr3dRZ8GiXra2lvB030sAnYjT82mzQzI8QWK4SgFNYocSSlMXbGyyVh-u_5CP-ha0VFdkfXAd-_lEzZfhbScx0tuZFa-F8PN4sIeegHtbPwGvrOlWqNLh61N9RuB4/s1600/social+networking.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUNQBuFhLd4QKSXXJr3dRZ8GiXra2lvB030sAnYjT82mzQzI8QWK4SgFNYocSSlMXbGyyVh-u_5CP-ha0VFdkfXAd-_lEzZfhbScx0tuZFa-F8PN4sIeegHtbPwGvrOlWqNLh61N9RuB4/s1600/social+networking.jpg" height="149" width="200" /></a></div>
Social Networking</h3>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
You might not think social networking plays a part here. Ever hear of <a href="https://github.com/" target="_blank">github</a>? Github drastically changed the way people code by making it cool and fun. Think of facebook, but for coders. The addition of the concept of merge/pull requests forces social interaction and code review. There is no better way to increase code quality than to have developers socialize and discuss the code in real-time.<br />
<br /></div>
<h3 style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmdlbLpABvMIAY73z5L6Dv48PBoYW_w4NYm99GcTx8fqGN3stYxgesNU7Xt0zlyFfB6diMYixf4tYgVYGBMLyKzx7BS22jZSPED54Lu6RuWau3-Z_P4VTJlG5tdAFbPS-zXUwjcxC1dfA/s1600/tdd.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmdlbLpABvMIAY73z5L6Dv48PBoYW_w4NYm99GcTx8fqGN3stYxgesNU7Xt0zlyFfB6diMYixf4tYgVYGBMLyKzx7BS22jZSPED54Lu6RuWau3-Z_P4VTJlG5tdAFbPS-zXUwjcxC1dfA/s1600/tdd.jpg" height="188" width="200" /></a></div>
Test Driven Development</h3>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<span style="line-height: 1.6;">I have been combining a number of open source projects to find a quick way to build an end-to-end platform for continuous integration and development (CI/CD). The use case was so I could embed CI/CD into my development work while also incorporating linux namespaces to save on compute and memory resources so I could run it locally, as well as for cost savings from providers such as AWS, DigitalOcean, On-Premise, etc when running a "production" version. </span><br />
<span style="line-height: 1.6;"><br /></span>
<span style="line-height: 1.6;">I want to push code, have it run tests and show me the output in real time, when tests pass it should restart my web service with the new code. Sounds simple, right? Actually, when you use SaltStack, Docker and Open Virtual Switch, it is!</span></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<br /></div>
<div style="box-sizing: border-box; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
<div style="color: #333333;">
<span style="line-height: 1.6;">Using the following open source projects, I was able to create my Developer environment not only locally, but also in the cloud with less than anticipated cost: </span><a href="https://about.gitlab.com/" style="line-height: 1.6;" target="_blank">GitLab</a><span style="line-height: 1.6;">, </span><a href="http://www.saltstack.com/" style="line-height: 1.6;" target="_blank">SaltStack</a><span style="line-height: 1.6;">, </span><a href="https://www.docker.com/" style="line-height: 1.6;" target="_blank">Docker</a><span style="line-height: 1.6;">, </span><a href="http://openvswitch.org/" style="line-height: 1.6;" target="_blank">OpenvSwitch</a><span style="line-height: 1.6;">, </span><a href="https://github.com/jpetazzo/pipework" style="line-height: 1.6;" target="_blank">Pipework</a><span style="line-height: 1.6;">, and </span><a href="http://mesosphere.io/" style="line-height: 1.6;" target="_blank">Mesosphere</a><span style="line-height: 1.6;">.</span></div>
<div style="color: #333333;">
<span style="line-height: 1.6;"><br /></span></div>
<div style="color: #333333;">
<span style="line-height: 1.6;"><br /></span></div>
</div>
<div class="separator" style="clear: both; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEewtAV2_rRxej6lZO0qu3m-HR8o15cUyLsSCs9-jTsUa0RPFjNzwDrg3Rdn0d1sA8ZXZFJUO9tOcU8k97Z1H3aqVXgKjPtfWAAoe5keJJjEnMBSmRZyff42IyUQ8IT_VyDtRk925ROWw/s1600/cicd.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEewtAV2_rRxej6lZO0qu3m-HR8o15cUyLsSCs9-jTsUa0RPFjNzwDrg3Rdn0d1sA8ZXZFJUO9tOcU8k97Z1H3aqVXgKjPtfWAAoe5keJJjEnMBSmRZyff42IyUQ8IT_VyDtRk925ROWw/s1600/cicd.jpg" height="162" width="400" /></a></div>
<h3 id="the-journey" style="box-sizing: border-box; color: inherit; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 1.5em; font-weight: 500; line-height: 1.1; margin-bottom: 9px; margin-top: 35px; position: relative;">
The Journey<a href="https://git.madisonhub.org/salt/digitalocean/wikis/brief-update#the-journey" style="background: transparent; box-sizing: border-box; color: #446e9b; margin-top: 0px; outline: none; text-decoration: none;"></a></h3>
<div style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
The journey began with a simple goal of building a development environment using Vagrant and have it run in much the same way a production system would. With my personal introduction to SaltStack, Software Defined Networking and Docker containers the journey turned into a twist of configuration management, cluster orchestration, resource management, containers, and software defined networking.</div>
<div style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
<div style="box-sizing: border-box; margin-bottom: 9px;">
<b>The goal</b> was to create a super project centered on vagrant that was reusable and could be deployed in minutes. It had to be configured to mimic provider environments such as <a href="http://aws.amazon.com/" target="_blank">AWS</a> and <a href="https://www.digitalocean.com/" target="_blank">Digital Ocean</a> and on-premise. It needed to be deploy, configured and run end-to-end locally as well as on any provider with the same effort and ease as issuing <code style="background-color: #f9f2f4; border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; border-top-left-radius: 4px; border-top-right-radius: 4px; box-sizing: border-box; color: #555555; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: inherit; font-weight: inherit; margin-top: 0px; padding: 0px 4px; white-space: nowrap;">vagrant up</code>.</div>
<div style="box-sizing: border-box; color: black; margin-bottom: 9px;">
<span style="line-height: 1.6;"><i>End-to-end implies that the solution deploys, configures, and starts up a complete infrastructure to develop applications using continuous integration and development. This applies to any provider in the cloud or on-premise.</i></span><br />
<br />
<br /></div>
</div>
<div style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipWY8IOW-_Y8zyzQ_HsQcHv_BF8EVh1MrUY8hMLTaTi5Bwq487EDuhrSA5nZvpXZYkPpJUYm4w9e4yooV-KX5cCnoAGXmDVwDBZzOdvDfOz__w49jGki3v1IeUPYJqPoYEI3jHK4EM0PA/s1600/vagrant.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipWY8IOW-_Y8zyzQ_HsQcHv_BF8EVh1MrUY8hMLTaTi5Bwq487EDuhrSA5nZvpXZYkPpJUYm4w9e4yooV-KX5cCnoAGXmDVwDBZzOdvDfOz__w49jGki3v1IeUPYJqPoYEI3jHK4EM0PA/s1600/vagrant.jpg" height="200" width="163" /></a></div>
<h5 id="enter-vagrant" style="box-sizing: border-box; color: inherit; font-size: 13px; line-height: 1.1; margin-bottom: 9px; margin-top: 9px; position: relative;">
<span style="box-sizing: border-box; margin-top: 0px;"><span style="font-size: 14px; line-height: 1.6;">Enter <a href="http://www.vagrantup.com/" target="_blank">Vagrant</a>:</span></span></h5>
The <a href="http://docs.vagrantup.com/v2/vagrantfile/index.html" target="_blank">Vagrantfile </a>stores useful building steps for an environment and its application. It typically instructs vagrant to build a virtual machine similar to a production environment. My blog post on <a href="http://portalstack.blogspot.com/2013/11/vagrant-virtualbox-ubuntu-for-linux.html" style="background: transparent; box-sizing: border-box; color: #446e9b; margin-top: 0px; outline: none; text-decoration: none;">Vagrant + VirtualBox + Ubuntu for linux development</a> details working with Vagrant for locally running a development branch of an application. Since then, I've put a focus on scaling containers with as little effort as possible. Scale in this scenario means having docker containers communicate with each other across <i>n </i>docker hosts where <i>n </i>is an arbitrary number greater than one. <br />
<br />
<b><br /></b>
<b><br /></b>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-Ps_KhkZ5EWM26Znu9rJTKZw6CirkkcoYxI7oKJVh6DVxd4YNHrMKG-LS06iQdJL4x9JJ5h60t59PMGw42UK0y8owoq55E7umlgVdzRYaKGBloWXE6sPOaXJbvSi8eXREihPSQqvjue8/s1600/docker.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-Ps_KhkZ5EWM26Znu9rJTKZw6CirkkcoYxI7oKJVh6DVxd4YNHrMKG-LS06iQdJL4x9JJ5h60t59PMGw42UK0y8owoq55E7umlgVdzRYaKGBloWXE6sPOaXJbvSi8eXREihPSQqvjue8/s1600/docker.jpg" /></a></div>
<b>Enter <a href="https://docker.io/" target="_blank">Docker</a>:</b><br />
When I first started using docker containers to consolidate servers, I ran into the issue of streamlining them because of the need to pipe and forward ports. This caused complexity in a number of services including riak clusters, percona clusters, and other cluster type services. There was no straight forward pre-baked solution to pipe all the necessary components together across docker hosts. Therefore it was difficult creating a completely containerized environment for certain services.<br />
<br />
<b><br /></b>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh07_Cto-cOxhccEXXpNuL0R3GKBqoBmbnlRi-iJMmA_ayv7Y6O8_l2vYE6kYJD0DJEB6AzLdj3RL5KJzNdzAh4ILWrIAjnAZEN_0xVrn7_nq6-6ZjQc6oTZhQ0GvFjVLQAJe4KWa1RF6Q/s1600/serf.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh07_Cto-cOxhccEXXpNuL0R3GKBqoBmbnlRi-iJMmA_ayv7Y6O8_l2vYE6kYJD0DJEB6AzLdj3RL5KJzNdzAh4ILWrIAjnAZEN_0xVrn7_nq6-6ZjQc6oTZhQ0GvFjVLQAJe4KWa1RF6Q/s1600/serf.jpg" /></a></div>
<b><br /></b>
<b>Enter Service Discovery:</b><br />
With the introduction of scale to the overall design, containers needed a way to easily communicate with each other across docker hosts, especially for database clusters and web applications with multiple components. Many service discovery projects started popping up on github. Some examples include <a href="https://github.com/crosbymichael/skydock" target="_blank">skydock</a>, <a href="https://github.com/coreos/etcd" target="_blank">etcd</a>, and basic Docker patterns such as <a href="https://docs.docker.com/articles/ambassador_pattern_linking/" target="_blank">ambassador</a> linking or even just port forwarding.<br />
<br />
The problem I have with this shift, is that its not solving the actual problem. A new problem is created because communication is now a crazy rats nest of port forwarding and a reliance on yet another service which could fail that keeps track of the information. Besides, how do you scale service discovery? Scaling service discovery, now hat's a whole issue itself.<br />
<br />
<b>My Path to Service Discovery:</b><br />
I believe we already have a reliable service discovery technology stack. It exists in the networking layer! Combine mac addresses with DHCP to get <a href="http://en.wikipedia.org/wiki/IP_address" target="_blank">IP addresses</a> matched with <a href="http://en.wikipedia.org/wiki/Address_Resolution_Protocol" target="_blank">DNS</a> entries and your service discovery is an <a href="http://en.wikipedia.org/wiki/Address_Resolution_Protocol" target="_blank">arp</a> table, available to any device attached to the network. Now, instead of re-inventing service discovery, we can simply "grease the wheel" by adding logic to Software Defined Networking such as customizing network flow and flow rates. <br />
<br />
I chose to take a different path conceptualizing service discovery by using Software Defined Networking. Just like virtual machines run inside physical machines, SDN lets you create networks inside networks. Pretty cool, huh? Open vSwitch, in particular, has support for tunnel bridges, meaning you can virtually plug servers into each other like one big network switch. Then, you add docker containers to this bridged network, and they all communicate over layer 2 networking. No port forwarding or central discovery service required. All instances (virtual and containerized) that have an interface on the virtual switch bridge can talk to each other.<br />
<br />
<b>Enter SaltStack</b>:<br />
<a href="https://github.com/saltstack/salt" target="_blank">SaltStack</a> is a great project. Defined on its <a href="http://www.saltstack.com/" target="_blank">website</a>, its a "<i>Fast, scalable and flexible systems management software for data center automation, cloud orchestration, server provisioning, configuration management and more</i>". It's a configuration management system built on ZeroMQ and written in python. Recipes are written in yaml and the <a href="http://en.wikipedia.org/wiki/Domain-specific_language" target="_blank">DSL</a> is easy to pick up. It's extremely modular and supports many different aspects of linux systems. State recipes provide a way to daisy chain installation and configuration of dependencies such as packages, services, files, etc for service components such as Docker and Open vSwitch.<br />
<br />
I now use SaltStack with Vagrant to automatically enforce state recipes on a schedule which run simple bash scripts to setup and maintain the health of an Open vSwitch network and its bridges as well as dependencies for docker and its services. So, with salt I can deploy an end-to-end environment for CI/CD, but I still need to pre-determine which docker host my containers run on as well as keep an eye on resource utilization and manually shuffle containers around if needed. Besides that, containers always use the same IP so if I needed to re-deploy a component, it would be available once the ARP table was updated.<br />
<br />
I started looking around at existing multi-host compute systems and realized that big data and compute clusters was exactly what I needed to magically place docker containers where they needed to go based on rules and resources.<br />
<br />
<b>Resource Management and Orchestration:</b><br />
Where docker containers end up being run is pre-determined and written to salt recipes for the most effective stacking. This allows for iterative changes and additions when scale is needed. The only problem with this approach is the complexity of the switch interconnects defined in open vSwitch. As the number of docker hosts increase the number of interconnects increases exponentially. Having a large number of interconnects is not the problem, manually writing them is. <br />
<br />
This is currently where I am in my journey and after looking around for a while, I think Mesos and its <a href="https://github.com/mesosphere/deimos" target="_blank">deimos</a> plugin for docker containers fits the bill quite nicely. <a href="https://github.com/apache/mesos" target="_blank">Mesos </a>is a cluster technology that supports pretty much all existing cluster engines including <a href="http://hadoop.apache.org/" target="_blank">hadoop</a>, google's <a href="https://github.com/GoogleCloudPlatform/kubernetes" target="_blank">kubernetes</a>, and others. <br />
<br />
What do I want to do now? I want to incorporate Open vSwitch and <a href="https://github.com/jpetazzo/pipework" target="_blank">pipework</a> in some way so that bridged networking can be deployed with containers kind of like how <a href="http://www.openstack.org/" target="_blank">openstack</a> works with virtual machines and their networks.<br />
<h2>
The Prototypes</h2>
</div>
<h4 id="vagrant-prototype-v1" style="box-sizing: border-box; color: inherit; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 1.2em; font-weight: 500; line-height: 1.1; margin-bottom: 9px; margin-top: 30px; position: relative;">
Vagrant Prototype v1 <a href="https://git.madisonhub.org/salt/digitalocean/wikis/brief-update#vagrant-prototype-v1" style="background: transparent; box-sizing: border-box; color: #446e9b; margin-top: 0px; outline: none; text-decoration: none;"></a></h4>
<div style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
I started with my first prototype inside Vagrant on my local workstation. I wasn't ready to throw anything up in the cloud because I wasn't sure how many resources I would need and thus how much it would cost me. <br />
<br />
As things progressed and all the networking was configured properly, I began to notice that I could run all the services I needed for the full life cycle of an application with very little resources (as low as 4GB of RAM). This included nagios monitoring, the java based kibana logging stack (elasticsearch and logstash), GitLab, GitLab-CI, salt-master and the Open vSwitch network itself.<br />
<br />
<span style="line-height: 1.6;">Network saturation was low. Overall cpu load was also low. The only thing that I had to keep an eye on was memory utilization. Even that was alleviated with a little bit of swap and proper planning.</span><br />
<span style="line-height: 1.6;"><br /></span>
<span style="line-height: 1.6;">I could clearly run this on Digital Ocean for around $50/mo no problem. </span></div>
<div class="separator" style="clear: both; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLG0i5QSph1NRkAwvPoqYK9kKxFJ2RfkQGGhzam-Dn1eQdUz9bB8oAvXXpKB6Y2VY-VS0D9NWsrJ2FAmP9MaEEej93CKPffoDcT2VOOblBXeUimCL6l8odAXYwArN9hPSzHZXTWvgmXyE/s1600/2host-4vm.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLG0i5QSph1NRkAwvPoqYK9kKxFJ2RfkQGGhzam-Dn1eQdUz9bB8oAvXXpKB6Y2VY-VS0D9NWsrJ2FAmP9MaEEej93CKPffoDcT2VOOblBXeUimCL6l8odAXYwArN9hPSzHZXTWvgmXyE/s1600/2host-4vm.png" height="326" width="400" /></a></div>
<h4 id="digital-ocean-prototype-v2" style="box-sizing: border-box; color: inherit; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 1.2em; font-weight: 500; line-height: 1.1; margin-bottom: 9px; margin-top: 30px; position: relative;">
Digital Ocean Prototype v2<a href="https://git.madisonhub.org/salt/digitalocean/wikis/brief-update#digital-ocean-prototype-v2" style="background: transparent; box-sizing: border-box; color: #446e9b; margin-top: 0px; outline: none; text-decoration: none;"></a></h4>
<div style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
The first cloud prototype was launched on Digital Ocean and worked flawlessly. With the use of UFW and changing the SSH ports, the environment was, for the most part, secure and safe from outside trouble. All cluster networking was contained to just the private network which was physically located inside the data center. There was a limitation, though. The private networking did not span across data centers so there was no way to securely enable multi-zone high availability. Servers would go down bringing services down with them, but because of the way containers were spread out, impact was minimal and services were quick to restore.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikrq0fjlen5sscl614WqYvKmLnKXj8GXROAUUJBg7obc7xRvVwb0lpki183NWYh33RGh0UHss1-btmZDj6TW-VbfS4iSxdG0ddTb8FvUQYw8Of2sg7KkmBfgZSEmIxST7fvAEomHhbtRo/s1600/Screen_Shot_2014-07-09_at_9.28.36_AM+(3).png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikrq0fjlen5sscl614WqYvKmLnKXj8GXROAUUJBg7obc7xRvVwb0lpki183NWYh33RGh0UHss1-btmZDj6TW-VbfS4iSxdG0ddTb8FvUQYw8Of2sg7KkmBfgZSEmIxST7fvAEomHhbtRo/s1600/Screen_Shot_2014-07-09_at_9.28.36_AM+(3).png" height="587" width="640" /></a></div>
<br /></div>
<h4 id="aws-prototype-v3" style="box-sizing: border-box; color: inherit; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 1.2em; font-weight: 500; line-height: 1.1; margin-bottom: 9px; margin-top: 30px; position: relative;">
AWS Prototype v3<a href="https://git.madisonhub.org/salt/digitalocean/wikis/brief-update#aws-prototype-v3" style="background: transparent; box-sizing: border-box; color: #446e9b; margin-top: 0px; outline: none; text-decoration: none;"></a></h4>
<div style="box-sizing: border-box; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 9px;">
The latest working prototype was launched on AWS after <a href="http://aws.amazon.com/about-aws/whats-new/2014/07/01/introducing-t2-the-new-low-cost-general-purpose-instance-type-for-amazon-ec2/" target="_blank">new EC2 pricing was announced</a>. The pricing is actually cheaper than Digital Ocean, so it was a no brainer to move back over and take advantage of virtual private cloud (VPC) networks. <b>VPC is Amazon's own implementation of software defined networking</b>, allowing you to create a restricted environment not exposed to the outside world. This allowed me to create a multi-zone highly available docker cluster completely isolated from the public internet. The only exposure into the cluster is through ssl load balancers, and only to expose specific services on specific ports.</div>
<div>
<div class="separator" style="clear: both; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; text-align: center;">
</div>
<div class="separator" style="clear: both; color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9SsKOJ_6z2A84pcpQRNUTq3CMpPMFM9Jh-2ymhTKujExCGEjGmqBy22VPYC8c6TEghZEkD0qxkBVOf8mugUIkqFQJ5pAohBE7E5vUBida8GmKprlfxgGcCgV_jbPpLGKeLobjSI13EV4/s1600/aws_map.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9SsKOJ_6z2A84pcpQRNUTq3CMpPMFM9Jh-2ymhTKujExCGEjGmqBy22VPYC8c6TEghZEkD0qxkBVOf8mugUIkqFQJ5pAohBE7E5vUBida8GmKprlfxgGcCgV_jbPpLGKeLobjSI13EV4/s1600/aws_map.jpg" height="358" width="640" /></a></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<br /></div>
<h3 style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
Whats Next?</h3>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
After a number of successful prototypes and months of rock solid stability in terms of uptime and network performance, I decided to move up on the technology stack and focus on auto provisioning and management of compute resources. At some point, manually adding hosts will become a challenge due to the exponential increase of switch interconnects. This is not as complicated as a problem as service discovery, however. A simple algorithm can be written to deal with switch interconnects.. </div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<br /></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<span style="line-height: 1.6;">This is where Mesosphere comes in. Mesos has plugins for spawning docker containers. Marathon is a simple interface which allows you to specify how much cpu, memory, and number of containers to create for a particular application. It then goes out and spawns a container where it fits resource wise.</span></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<span style="line-height: 1.6;"><br /></span></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<span style="line-height: 1.6;">I want to figure out a way to extend Marathon and Mesos (particularly <a href="https://github.com/mesosphere/deimos" target="_blank">deimos</a>), so that creating containers includes steps for adding virtual networks (using pipework). </span></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.6;">
<span style="line-height: 1.6;"><br /></span></div>
<div>
<span style="color: #333333; font-family: Helvetica Neue, Helvetica, Arial, sans-serif;"><span style="font-size: 14px; line-height: 1.6;">I get really excited thinking about the potential applications of this sort of setup. I know there are a lot of "docker clusters" out there, but I feel like exploiting traditional networking </span><span style="font-size: 14px; line-height: 22.399999618530273px;">technologies</span><span style="font-size: 14px; line-height: 1.6;"> and building a cluster on SDN makes more sense.</span></span></div>
</div>
<div>
<span style="color: #333333; font-family: Helvetica Neue, Helvetica, Arial, sans-serif;"><span style="font-size: 14px; line-height: 1.6;"><br /></span></span></div>
<div>
<span style="color: #333333; font-family: Helvetica Neue, Helvetica, Arial, sans-serif;"><span style="font-size: 14px; line-height: 1.6;"><br /></span></span></div>
<div>
<span style="color: #333333; font-family: Helvetica Neue, Helvetica, Arial, sans-serif;"><span style="font-size: 14px; line-height: 22.399999618530273px;">Looking forward to writing more, as things progress. Until then.</span></span></div>
</div>
</div>
Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com2tag:blogger.com,1999:blog-5361461834637096019.post-61883874939897669282013-11-15T08:05:00.001-08:002013-11-15T08:08:19.252-08:00Vagrant + VirtualBox + Ubuntu for linux developmentThe combination of <a href="http://www.vagrantup.com/" target="_blank">Vagrant</a>, <a href="https://www.virtualbox.org/" target="_blank">VirtualBox</a> and <a href="http://www.ubuntu.com/" target="_blank">Ubuntu</a> allows for some interesting potential. First of all, its a simple way to build and deploy cloud images similar to <a href="http://aws.amazon.com/" target="_blank">Amazon Web Service</a>'s <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html" target="_blank">AMIs</a>, but all on your local machine. It is also capable of customizing the virtual machine settings through a configuration file leaving us the opportunity to create full linux development desktop experiences. Finally, it allows us to run linux containers (such as <a href="http://www.docker.io/" target="_blank">docker</a>) on Windows and OS X environments in an extremely simple way.<br />
<br />
The use case I will be showing in this post is for people who prefer bare metal installs of either Windows or OS X, but would like to have a full screen linux environment such as ubuntu running gnome. With Vagrant and VirtualBox, this is universally possible.<br />
<br />
First, please head to <a href="http://git-scm.com/" target="_blank">http://git-scm.com</a> and download the git installer. For windows, make sure to install the unix tools, its worth it. The installer for windows will get openSSH installed which is required by Vagrant.<br />
<br />
Next step: Install <a href="https://www.virtualbox.org/wiki/Downloads" target="_blank">VirtualBox</a>. Once virtual box is installed, also install the <a href="http://download.virtualbox.org/virtualbox/4.3.2/Oracle_VM_VirtualBox_Extension_Pack-4.3.2-90405.vbox-extpack" target="_blank">extension pack</a>.<br />
<br />
Finally, Install <a href="http://downloads.vagrantup.com/" target="_blank">Vagrant</a>. <br />
<br />
<br />
Once all the dependencies are installed, open up a shell window and lets get started. We will use a base cloud image from Ubuntu 13.04 found on <a href="http://vagrantbox.es/" target="_blank">http://vagrantbox.es</a>. Vagrant works in project directories, so first create a project folder. Inside the project folder we will initialize a default configuration file and start an instance. Once the instance is ready we will install ubuntu-gnome-desktop.<br />
<br />
<b><span style="font-family: Courier New, Courier, monospace;">$ cd ~</span></b><br />
<b><span style="font-family: Courier New, Courier, monospace;">$ mkdir ubuntudesktop; cd ubuntudesktop</span></b><br />
<b><span style="font-family: Courier New, Courier, monospace;"><br /></span></b>
<b><span style="font-family: Courier New, Courier, monospace;">$ vagrant box add ubuntu http://cloud-images.ubuntu.com/vagrant/raring/current/raring-server-cloudimg-amd64-vagrant-disk1.box</span></b><br />
<b><span style="font-family: Courier New, Courier, monospace;"><br /></span></b>
<b><span style="font-family: Courier New, Courier, monospace;">$ vagrant init</span></b><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<b><span style="font-family: Courier New, Courier, monospace;">$ vagrant up</span></b><br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCs6CMULSnN9TfN8m0CEJB30jxG7JP8iaQaEu4CX5E2fWGtHSxRFZnbjLxty223VdYbye-toPPrnpxmQIfJz5o_NPpmEtHbwGJQCabN7_9Krm_38sqyRXmNbEqZkTXYeDtYMF72kC39Qk/s1600/Screenshot+2013-11-15+08.46.12.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="497" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCs6CMULSnN9TfN8m0CEJB30jxG7JP8iaQaEu4CX5E2fWGtHSxRFZnbjLxty223VdYbye-toPPrnpxmQIfJz5o_NPpmEtHbwGJQCabN7_9Krm_38sqyRXmNbEqZkTXYeDtYMF72kC39Qk/s640/Screenshot+2013-11-15+08.46.12.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Once you issue vagrant up, a new virtual machine will be provisioned inside virtualbox. Once the instance loads, vagrant will complete its tasks. At this point we can use vagrant to SSH into the instance using public keys. <br />
<br />
<b><span style="font-family: Courier New, Courier, monospace;">$ vagrant ssh</span></b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1lsWa2b8uW1qqFx6B77lFwPuwJR6kQ1_JLbFxe6ygZbONHg40jFH6QVFT_EIcyI82kKfWHNdWf-Zr-AKPZDiAHIrlNgr1gYBA6g-ec8D3jdGXgMRb987UYGCtZN3QeGv94Bye0BT9770/s1600/Screenshot+2013-11-15+09.43.47.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1lsWa2b8uW1qqFx6B77lFwPuwJR6kQ1_JLbFxe6ygZbONHg40jFH6QVFT_EIcyI82kKfWHNdWf-Zr-AKPZDiAHIrlNgr1gYBA6g-ec8D3jdGXgMRb987UYGCtZN3QeGv94Bye0BT9770/s640/Screenshot+2013-11-15+09.43.47.png" width="640" /></a></div>
<br />
<br />
We have lift off. Now lets install the desktop environment. While inside the virtual machine instance through SSH, do the following:<br />
<br />
<b><span style="font-family: Courier New, Courier, monospace;">vagrant@vagrant-ubuntu-raring-64:~$ sudo apt-get update; sudo apt-get -y install ubuntu-gnome-desktop</span></b><br />
<br />
Wait for a while at this point. Its going to install quite a bit of stuff (1.6GB to be exact). Once it finishes, exit the ssh session. Now that we are back on the host terminal session, lets halt the virtual machine and replace its configuration file with a more appropriate desktop capable set of configurations.<br />
<br />
$ vagrant halt<br />
<br />
Edit the Vagrantfile located inside the current working directory, which should be your project folder. Replace its contents with the following:<br />
<br />
<br />
<script src="https://gist.github.com/mikekunze/7486548.js"></script>
Now that we have updated our configuration, start the instance back up and be amazed. Yesss.<br />
<br />
Now, login as vagrant (password vagrant). <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPnCu_zPVJyo-WxRQS_fe-pm0CNwAsmyou9VYx_vGUi4xIRkq8Xpma-tayNIxrWsG5KHu5OPoU1oZwI1-D893bQ6evTYHag92vXCcgHHgzh0Fkis7puiTSFqkI8-YXzl4C7v9HFX4v_rM/s1600/Screenshot+2013-11-15+09.53.44.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="428" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPnCu_zPVJyo-WxRQS_fe-pm0CNwAsmyou9VYx_vGUi4xIRkq8Xpma-tayNIxrWsG5KHu5OPoU1oZwI1-D893bQ6evTYHag92vXCcgHHgzh0Fkis7puiTSFqkI8-YXzl4C7v9HFX4v_rM/s640/Screenshot+2013-11-15+09.53.44.png" width="640" /></a></div>
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com4tag:blogger.com,1999:blog-5361461834637096019.post-52035610806360360952013-10-23T06:41:00.004-07:002013-11-06T07:34:40.776-08:00Docker on EC2 Micro and maintaining storage growthIf you use docker for integration or continuous testing, you may be re-building images. Every command issued through docker keeps a commit of the fs changes, so disk space can fill up fast; extremely fast on an EC2 micro instance with 8GB of EBS.<br />
<br />
Scrounging around the issues in the docker project on github, I ran across a thread talking about solutions for the storage growth. I took it and expanded a bit.<br />
<br />
Below is the output of past and present docker commands. Anything with a status of "Up for x minutes", those are presently running docker commands. Anything with Exit 0 or other Exit values are ended and can be discarded if they are not needed, such as you do not need to commit the changes to a new image. In the screenshot below, you can see a re-build of the mongodb image. There are two commands the Dockerfile issued that stored commits and they can be discarded.
<br />
<br />
$ sudo docker ps -a
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcG27Y9yURGKHrELpJRwzJVURnjVXokDwgvyomQecNvjE0PLC0vPEq5N9cNk_zjHZiGuOPIwOecvl38GaHOd6zUFaevYeBipWAfB-8_Wmcj8EXJbASF9WjFSWBfGkgfmSdXv9KRb7biTI/s1600/Screenshot+2013-10-23+08.26.17.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="44" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcG27Y9yURGKHrELpJRwzJVURnjVXokDwgvyomQecNvjE0PLC0vPEq5N9cNk_zjHZiGuOPIwOecvl38GaHOd6zUFaevYeBipWAfB-8_Wmcj8EXJbASF9WjFSWBfGkgfmSdXv9KRb7biTI/s640/Screenshot+2013-10-23+08.26.17.png" width="640" /></a></div>
<br />
<br />
<h3>
TO DELETE UNUSED CONTAINERS: </h3>
<br />
<b>$ sudo docker ps -a | grep 'Exit 0' | awk '{print $1}' | xargs docker rm
</b><br />
<br />
​This will find any container with a "I have no error" exit status and delete them. Note: There may be other exit statuses depending on how well an image build went. If some of the commands issued in the Dockerfile are bad or fail, the status field will have a different Exit value, so just update the piped grep command with that string.
<br />
<br />
<br />
<br />
<br />
<h3>
TO DELETE UNUSED IMAGES: </h3>
<br />
<b>$ sudo docker images | grep 'none' | awk '{print $3}' | xargs docker rmi</b><br />
<none><br /></none>
This will find any images that are not tagged or named. This is typical when an image is re-built. For example: if you have a running container based on an image that was re-built, `docker ps` will show that container with a hash for its image name. That just means the container is running an old (and referenced) image. Once you stop the container and replace it with a new one, running the above command will find it and remove it from the file system.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-7648050928067983672013-10-19T08:02:00.002-07:002013-10-19T08:13:14.457-07:00Closures, javascript and how<div>
From the <a href="http://en.wikipedia.org/wiki/Closure_(computer_science)" target="_blank">wiki article</a>, a closure (computer science) is a function or reference to a function together with a referencing environment-- a table storing a reference to each of the non-local variables of that function. </div>
<div>
<br /></div>
<div>
Closure-like constructs include callbacks and as such, are important in asynchronous programming. Here is a simple example in PHP that uses a closure as a callback to compute the total price of a shopping cart by defining a reference table for the callback function and including variables tax and total:</div>
<div>
<br />
<script src="https://gist.github.com/mikekunze/7056464.js"></script></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
The concept of closures in javascript is important to understand because you might not even know you're using it. If you write in coffee-script classes or do classical inheritance patterns in vanilla javascript or even write callbacks in general for asynchronous programming, you are probably using closures. The following example is a starting point for classical inheritance in javascript. This shows how to hide private variables. It doesn't use "new" but the pattern is very similar.<br />
<br />
<br />
<script src="https://gist.github.com/mikekunze/7056806.js"></script></div>
<div>
<br /></div>
<div>
According to <a href="http://www.amazon.com/Effective-JavaScript-Specific-Software-Development/dp/0321812182" target="_blank">Effective Javascript: 68 Specific Ways To Harness The Power of Javascript</a>, there are three essential facts regarding closures:</div>
<div>
<ul>
<li>JavaScript allows you to refer to variables that were defined outside of the current function</li>
<li>Functions can refer to variables defined in outer functions even after those outer functions have returned</li>
<li>Closures can update values of outer variables</li>
</ul>
<div>
<br /></div>
<div>
Knowing this, we can do some fun stuff in Node.JS with asynchronous programming. With closures, we can pull a document collection from a NoSQL database, manipulate the results, and push it to an array stored via closure in the parent scope.</div>
</div>
<div>
<br />
<script src="https://gist.github.com/mikekunze/7057021.js"></script></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Hopefully you will use closures to your advantage, especially when developing in javascript, be it server side or client side or even in the database (<a href="https://postgres.heroku.com/blog/past/2013/6/5/javascript_in_your_postgres/" target="_blank">Postgres with v8</a>).</div>
<div>
<br /></div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-32087795828481706322013-09-17T12:05:00.000-07:002013-09-17T12:05:19.796-07:00SharePoint 2013 - 503 Unavailable after March 2013 PUSharePoint 2013 recently had two cumulative updates released: March 2013 PU mandatory update, and the June 2013 CU. I wont go into the details of obtaining these patches or running them. Basically, you'll need a lot of down time to run the patches, as they take a while. With that said, lets assume the patches ran, updated and completed. Lets also assume you have already run the product configuration wizard after each patch to update the database.<br />
<br />
For me, I had to run the patch a few times. Due to randomness (or maybe just sloppy closed source coding) SharePoint CUs tend to fail. The good thing, is that they either succeed 100% or fail completely (leaving SharePoint more or less in an OK state. Luckily for me, I did not have any issues running the product configuration wizard. I've seen and heard of instances where product configuration wizard fails, but its usually a clean up task that fails and isnt a big deal.<br />
<br />
After I updated the March 2013 PU, I ran into a 503 unavailable for central admin and my site collections. Instead of finding an immediate resolution, I plowed forth and installed the June 2013 CU with success. Unfortunately after the database upgrade and a SharePoint server reboot, I was still getting a 503 error when trying to access SharePoint.<br />
<br />
After a bit of googling, I found a working solution to this particular problem. Load up IIS Manager and head to the Server Application Pools. As per <a href="http://www.rightpoint.com/community/blogs/viewpoint/archive/2011/11/07/http-503-service-unavailable-after-a-sharepoint-upgrade-help.aspx" target="_blank">this post</a> and for me, all of my Application Pools were stopped. Without hesitation, I started every stopped pool, restarted IIS and once again was able to access Central Admin and my Site Collections.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZTdTOX2Ku9NlEDvkROSxWk18-brtQ2iFDZyP_HMDFs18tL77LE0Ru9PXNVwEn0l6wjw5uXFOCWhPQG7Yh8jmozBN4Ah_jV63oIz65ktw8gugs3T7-F_lL6znaHiF1aIiiXFmzKu-bXWw/s1600/Screen+Shot+2013-09-17+at+2.03.06+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="534" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZTdTOX2Ku9NlEDvkROSxWk18-brtQ2iFDZyP_HMDFs18tL77LE0Ru9PXNVwEn0l6wjw5uXFOCWhPQG7Yh8jmozBN4Ah_jV63oIz65ktw8gugs3T7-F_lL6znaHiF1aIiiXFmzKu-bXWw/s640/Screen+Shot+2013-09-17+at+2.03.06+PM.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com37tag:blogger.com,1999:blog-5361461834637096019.post-57187247587227017952013-08-24T08:53:00.002-07:002013-08-24T08:53:35.922-07:00Using classical inheritance to contain express controllers and socket eventsCoffeescript has +1'ed itself for me lately. I will tell you why and try to explain. As far as I know, there is no native support in coffeescript for having a class extend multiple parents classes. In plain javascript, scope and prototypal inheritance can be confusing, as you'll see later on. <br />
<br />
Lets imagine we are taking our express web application and isolating out the app routes and the socket events. The reason for this is simple: when a Web class extends the controller and event classes, they can share important pieces of information related to the web server such as a user session or login information. This will help us later on when implementing socket events for authorized users.<br />
<br />
Here is the entire bit of code to get you going. It is simple, elegant, and for the most part completely readable as long as you have a bit of understanding in regards to the way prototypes in javascript work.<br />
<br />
<script src="https://gist.github.com/mikekunze/6328781.js"></script><br />
<br />
<br />
Here is the compiled, fully javascript compliant version. As you can see, its a bit messy and definitely not as readable as the coffeescript version.<br />
<br />
<script src="https://gist.github.com/mikekunze/6328790.js"></script><br />
<br />
Happy coding.Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-41006386702422444942013-08-19T15:22:00.000-07:002013-11-05T10:24:22.139-08:00Custom Nagios Notifications, now with Jade<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7vgQOgECASNQhAT0QGh0ZM4lgwiNLVTdW_Eai5e9d7aNdZgHmCLGOZWIZv5o7f_S1U62wRY1i5OyIOvQULRKIN60AKxi68R2zyaxua0Obpo4EBBC2S5AbWFq_KGi2QJk_3BGfRr9sLMY/s1600/Screen+Shot+2013-08-19+at+5.14.07+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7vgQOgECASNQhAT0QGh0ZM4lgwiNLVTdW_Eai5e9d7aNdZgHmCLGOZWIZv5o7f_S1U62wRY1i5OyIOvQULRKIN60AKxi68R2zyaxua0Obpo4EBBC2S5AbWFq_KGi2QJk_3BGfRr9sLMY/s400/Screen+Shot+2013-08-19+at+5.14.07+PM.png" width="400" /></a></div>
Recently, I <a href="http://portalstack.blogspot.com/2012/07/sending-custom-nagios-notification.html" target="_blank">posted</a> about using <a href="https://github.com/jashkenas/coffee-script" target="_blank">coffee-script</a> and <a href="https://github.com/andris9/Nodemailer" target="_blank">node-mailer</a> to send custom <a href="http://www.nagios.org/" target="_blank">Nagios</a> notification emails. While it is a great way of extending the capabilities of Nagios Alerts by allowing any arbitrary hook into other coffee / javascript apis (such as node-mailer), it is also capable of creating pretty HTML e-mails with the help of the <a href="http://jade-lang.com/" target="_blank">Jade</a> template engine.<br />
<br />
Since I've already created the Nagios Command and bound it to my contact information, all I need to do is install jade into the project folder's node_modules:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ npm install jade</span><br />
<br />
We then need to create our templates. Since I wanted to use a layout, I created a jade sub-folder in my projects folder.<br />
<br />
Here is the layout (./jade/layout.jade):<br />
<script src="https://gist.github.com/mikekunze/6274753.js"></script>
<br />
Here is a neat header bar that I'm using(./jade/nav-bar-email.jade):<br />
<script src="https://gist.github.com/mikekunze/6274764.js"></script>
<br />
Here is the alert e-mail(./jade/nagiosAlert.jade):<br />
<script src="https://gist.github.com/mikekunze/6274778.js"></script>
<br />
Here is what my notify-service.coffee script looks like(/opt/bin/notify-service.coffee):<br />
<script src="https://gist.github.com/mikekunze/6274708.js"></script>
<br />
<br />
The final result is a pretty looking E-Mail. I'm viewing this with Mail.app in OS X. Ive blanked out a few bits. Just imagine there is Company Text to the left of home and a link between home and projects in the nav-bar<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7vgQOgECASNQhAT0QGh0ZM4lgwiNLVTdW_Eai5e9d7aNdZgHmCLGOZWIZv5o7f_S1U62wRY1i5OyIOvQULRKIN60AKxi68R2zyaxua0Obpo4EBBC2S5AbWFq_KGi2QJk_3BGfRr9sLMY/s1600/Screen+Shot+2013-08-19+at+5.14.07+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="389" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7vgQOgECASNQhAT0QGh0ZM4lgwiNLVTdW_Eai5e9d7aNdZgHmCLGOZWIZv5o7f_S1U62wRY1i5OyIOvQULRKIN60AKxi68R2zyaxua0Obpo4EBBC2S5AbWFq_KGi2QJk_3BGfRr9sLMY/s640/Screen+Shot+2013-08-19+at+5.14.07+PM.png" width="640" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-46313029034697609772013-08-18T08:31:00.000-07:002013-08-18T08:31:38.889-07:00Simple Callbacks - Executing parallel and dependent tasks using async without the sphagetti<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaKEaHm1EmO5DOSz4PgyE5vKmvfGUaMu6a3kmuLuvIW1eu48IXTouIEiq3gt9VTLQ1zKLTSGMkA0Iy7qECsNI0ENMClZGzQ425kBxOeWo8f5RrRmjm_yAUVu4PAFInGCciN_hUoeC6mwY/s1600/IMG_1770.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaKEaHm1EmO5DOSz4PgyE5vKmvfGUaMu6a3kmuLuvIW1eu48IXTouIEiq3gt9VTLQ1zKLTSGMkA0Iy7qECsNI0ENMClZGzQ425kBxOeWo8f5RrRmjm_yAUVu4PAFInGCciN_hUoeC6mwY/s320/IMG_1770.JPG" width="239" /></a><span style="background-color: white; color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 14px; line-height: 19px;">There has been some bloggers complaining lately about the callback pattern and its spaghetti code structure. Some even compare it to <a href="http://tirania.org/blog/archive/2013/Aug-15.html" target="_blank">GOTO statements</a>, although that post is less about coffee/javascript. The beauty of javascript, especially written in coffee-script, is that you can conceptualize parallel and dependent tasks based on code indentation. Commands executed in the same depth are executed at the same time, while code resting deeper (in the particular case of callback patterns) are deferred until dependent execution is complete.</span><br />
<span style="background-color: white; color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 14px; line-height: 19px;"><br /></span>
<span style="background-color: white;"><span style="color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif;"><span style="font-size: 14px; line-height: 19px;">The following image on the left depicts what I am explaining above. Blue lines are executed at the same time. Red lines are dependent on their blue parent's completion before they execute, and so on.</span></span></span><br />
<span style="background-color: white;"><span style="color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif;"><span style="font-size: 14px; line-height: 19px;"><br /></span></span></span>
<span style="background-color: white; color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 14px; line-height: 19px;">I then take the idea a step further and mix batches of parallel tasks with tasks dependent on the batch to complete.</span><br />
<span style="background-color: white; color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 14px; line-height: 19px;"><br /></span>
<span style="background-color: white; color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 14px; line-height: 19px;">This is a sample gist of personal work I am doing on a Project Management webApp. The goal of this gist is to show how non dependent tasks can be parallelized and dependent tasks can be run after those parallel dependencies are run. eachTask iterator takes a task object and a callback. It uses the Function prototype method call to pass scope to each parallel task.</span><br />
<span style="background-color: white; color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 14px; line-height: 19px;"><br /></span>
<span style="background-color: white; color: #666666; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 14px; line-height: 19px;"><br /></span>
<br />
<script src="https://gist.github.com/mikekunze/6258884.js"></script>Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-11070128423874038152013-08-06T07:59:00.000-07:002013-08-06T07:59:50.432-07:00Async HTML5 File Uploads with Node / Express Framework<div>
File uploads with HTML5 these days is easy as pie, especially when you have a framework that handles everything for you, internally. Combining <a href="http://nodejs.org/" target="_blank">Node</a>, the <a href="http://expressjs.com/" target="_blank">Express</a> framework and a little HTML5 magic with <a href="https://developer.mozilla.org/en-US/docs/Web/API/FormData" target="_blank">FormData</a> and <a href="http://www.w3.org/TR/XMLHttpRequest/" target="_blank">XMLHttpRequest</a>, we can create a very simple file uploader that can support large files.</div>
<div>
<br /></div>
The Express framework supports <a href="http://expressjs.com/api.html#req.files" target="_blank">multi-part uploads</a> through its bodyParser middleware which utilizes the node-formidable module internally.<br />
<div>
<br /></div>
<div>
<br /></div>
<div>
<br />
Start off with the client side.</div>
<div>
<script src="https://gist.github.com/mikekunze/6155840.js"></script></div>
<div>
<br /></div>
<div>
<br />
Add the button click event with a bootstrap progress indicator
<script src="https://gist.github.com/mikekunze/6155928.js"></script>
<br />
<br />
<br />
<br />
Add the server route.<br />
<script src="https://gist.github.com/mikekunze/6156025.js"></script></div>
<br />
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-4969409698023210442013-07-11T16:09:00.000-07:002013-07-11T16:09:51.438-07:00Basic CKEditor 4 Image Uploader with Express and NodeJS<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgQ0XN46g1EAKahRc0xGieE8zV6blotgG0BTbtLAjhPKeEdh_kO_X0fiSYq1OxGBHQsGvZhGs7jX0SoBR_DlW2o-KTSzQfjZJ0NiYn95bI9ugBAAcjQUqur_0Hnns-6Os6QWdd93Leuj4/s1600/Screen+Shot+2013-07-11+at+5.33.22+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgQ0XN46g1EAKahRc0xGieE8zV6blotgG0BTbtLAjhPKeEdh_kO_X0fiSYq1OxGBHQsGvZhGs7jX0SoBR_DlW2o-KTSzQfjZJ0NiYn95bI9ugBAAcjQUqur_0Hnns-6Os6QWdd93Leuj4/s640/Screen+Shot+2013-07-11+at+5.33.22+PM.png" width="640" /></a></div>
<br />
<br />
I recently found an interesting <a href="http://www.caeus.com/articles/how-to-add-and-upload-an-image-using-ckeditor/" target="_blank">article</a> on a simple hack to get the existing image upload feature in <a href="http://ckeditor.com/download" target="_blank">CKEditor</a> 3 enabled and functioning with a <a href="http://php.net/" target="_blank">PHP</a> server. I took his idea and applied it to the latest version, which is currently 4 with a NodeJS and Express framework backend. <br />
<br />
It basically requires editing two files inside the ckeditor sdk: image.js and config.js.<br />
<br />
Edit ckeditor/plugins/image/dialogs/image.js and look for "editor.config.filebrowserImageBrowseLinkUrl" around line 975. A few lines below will be "hidden: true". Change this to "hidden: false". Further down is another "hidden: true" near "id: 'Upload'", which needs to be changed to "hidden: false". Once you are done with the changes, image.js should look like this.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDlJiZlIlgbYijQIN_8pkqBr3ByO9OkNrjTFvFAR9-xCJGIEqroaGf-fGy4j4Z1ZfKcQHlI3oTL2r6KYJJTM6ZwkVyzEdyqHAGPZXCsw4cIlrSM_ShlBZMWpEDtCIXJXtBRhovr2Jm8gw/s1600/Screen+Shot+2013-07-11+at+5.45.07+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDlJiZlIlgbYijQIN_8pkqBr3ByO9OkNrjTFvFAR9-xCJGIEqroaGf-fGy4j4Z1ZfKcQHlI3oTL2r6KYJJTM6ZwkVyzEdyqHAGPZXCsw4cIlrSM_ShlBZMWpEDtCIXJXtBRhovr2Jm8gw/s1600/Screen+Shot+2013-07-11+at+5.45.07+PM.png" /></a></div>
<div style="text-align: left;">
Next, we need to edit the config.js file to point to where the upload POST route is. Edit <b>ckeditor/config.js</b> and add <span style="font-family: Courier New, Courier, monospace;">config.filebrowserUploadUrl = '/upload'</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div style="text-align: left;">
<br /></div>
<br />
Next, we need to create our Express POST route to handle the upload. I am taking the temp file name and prepending it to the actual file name and saving it under ./public/uploads. Since public is a default static route in Express, any uploaded image will be immediately available in the CKEditor UI. The important part here is to return a script block, instructing CKEditor to take the new image.<br />
<br />
<br />
<br />
<script src="https://gist.github.com/mikekunze/5980077.js"></script>
<br />
<br />
<br />
Finally, route it through express:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">var fn = require("./upload.js");</span><br />
<span style="font-family: Courier New, Courier, monospace;">app.post("/upload", fn);</span>Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com62tag:blogger.com,1999:blog-5361461834637096019.post-13558638839367035752013-06-13T16:36:00.001-07:002013-07-27T07:57:53.835-07:00AWS Coffee-Script Web Stack - Part 1: The Development Stack<h2>
AWS Coffee-Script Web Stack - Part 1: The Development Stack</h2>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBiWJBCywjQWAKuC7ua9QGomL5McSOr2FxstUqzcduuWUaIfZdYsbzgNwhs-GnchwaIB3YMy0bg5iAqdHFvdffqHoApcaJFzNA5BN32UTR_ESNE6agCBPvu4pc3mQk9j2anpFkWRwwSuM/s1600/images.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="87" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBiWJBCywjQWAKuC7ua9QGomL5McSOr2FxstUqzcduuWUaIfZdYsbzgNwhs-GnchwaIB3YMy0bg5iAqdHFvdffqHoApcaJFzNA5BN32UTR_ESNE6agCBPvu4pc3mQk9j2anpFkWRwwSuM/s320/images.jpeg" width="320" /></a></div>
<div>
Welcome to a multi-part series on setting up an <a href="http://aws.amazon.com/" target="_blank">Amazon Web Service</a> <a href="http://coffeescript.org/" target="_blank">Coffee-Script</a> Web Server using <a href="http://nodejs.org/" target="_blank">NodeJS</a> and the <a href="http://expressjs.com/">Express</a> framework.</div>
<div>
<br /></div>
<div>
Part 1 of this series consists of building up the server components in a nice stack on top of ubuntu linux. By the end of this blog entry, you will have a basic personal website with the <a href="http://twitter.github.com/bootstrap/index.html" target="_blank">twitter bootstrap</a> look and feel. We will then expand on its capabilities and features as I write more posts on the series. Some of these posts will include: adding <a href="https://github.com/adunkman/connect-assets" target="_blank">connect-assets</a> which allows you to include compiled coffee-script into the rendered templates, session management with <a href="http://redis.io/" target="_blank">redis-server</a> for persistent session state across server reboots using oAuth providers such as google, combining sessions with <a href="http://socket.io/">socket.io</a> for seamless authentication validation, and simple M(odel) V(iew) C(ontroller) structure with classical inheritance design.</div>
<div>
<br /></div>
<div>
<b>The stack</b> will consist of three main <a href="http://nodejs.org/" target="_blank">NodeJS</a> components: node modules, server code, and client code. It will run on two environments: Internet facing AWS running Ubuntu Server 12.04 and a development environment consisting of Ubuntu Desktop 13.04 with <a href="http://www.jetbrains.com/" target="_blank">JetBrains</a> <a href="http://www.jetbrains.com/webstorm/" target="_blank">Webstorm</a> 6 Integrated Development Environment (IDE). The code will be stored in a <a href="http://git-scm.com/" target="_blank">git</a> repository which we use to keep a history of all our changes.<br />
<br />
<br />
<br />
<h4>
Setup Development Environment</h4>
So, lets get going and setup our development environment. First, <a href="http://www.ubuntu.com/download/desktop">download</a> the latest Ubuntu Linux Desktop, currently 13.04. While you wait for the ISO, go ahead and start building the Virtual Machine instance. The following screens relate to my Virtual Host based on <a href="http://www.vmware.com/products/fusion/overview.html" target="_blank">VMWare Fusion</a> 5.<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdt6lHxxo5ahi906i72gHDDXreYtOoEv7twIawD-XE8pCwUeCd2m5zzjRp27sFKTXHRskyCCjkJlDhskffPClxfZ2lvH8rkkMF3dmGIzOQvcJOWWIJAcFtflSS72mN88JF0-p_ATIRLuw/s1600/Screen+Shot+2012-12-23+at+1.44.46+PM.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="235" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdt6lHxxo5ahi906i72gHDDXreYtOoEv7twIawD-XE8pCwUeCd2m5zzjRp27sFKTXHRskyCCjkJlDhskffPClxfZ2lvH8rkkMF3dmGIzOQvcJOWWIJAcFtflSS72mN88JF0-p_ATIRLuw/s400/Screen+Shot+2012-12-23+at+1.44.46+PM.png" width="400" /></a></div>
<b>Virtual Hardware</b><br />
The minimum to get going smoothly is pretty lightweight compared to other environments such as <a href="http://sharepoint.microsoft.com/en-us/preview/sharepoint.aspx" target="_blank">Microsoft's SharePoint 2013</a> which requires a good 24GB of ram for a single development environment. This virtual machine should have at least 2 vCPU and 2GB RAM. Depending on how big your project gets (how many files Webstorm needs to keep an eye on), you might need to bump up to 4GB of RAM<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4-wwg3Z7ycF3U7GW6uaQkbyCvnuC5FFlL_xFL-sTGKC7z0_4KcqQyzk6T1DqlrXM_MsTxcDqojzTWZLt7KhQ4rmzcRJ56SLqfHDTCQ001UY_s8kb2dneo23t5DR5ChFXxr43n6TrsLN0/s1600/Screen+Shot+2012-12-23+at+1.50.19+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="91" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4-wwg3Z7ycF3U7GW6uaQkbyCvnuC5FFlL_xFL-sTGKC7z0_4KcqQyzk6T1DqlrXM_MsTxcDqojzTWZLt7KhQ4rmzcRJ56SLqfHDTCQ001UY_s8kb2dneo23t5DR5ChFXxr43n6TrsLN0/s400/Screen+Shot+2012-12-23+at+1.50.19+PM.png" width="400" /></a></div>
<br />
<b>OS Volume</b><br />
For OS volume, 20GB should suffice. If you plan on making upload projects, then provision more storage. <br />
<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhF-jUoZw2gTbTF2MrEwo-4njidQt61vGZrU7Gr3ywpouZsoLguYXu7Ojd1urY0aEcanOfIFm_j0x_2putvCz9B1HKj6PqbmymjxKLigTCV_zuph751xBF_IJVW9O2o4vK8yqIxxR3rTW4/s1600/Screen+Shot+2012-12-23+at+1.39.05+PM.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="311" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhF-jUoZw2gTbTF2MrEwo-4njidQt61vGZrU7Gr3ywpouZsoLguYXu7Ojd1urY0aEcanOfIFm_j0x_2putvCz9B1HKj6PqbmymjxKLigTCV_zuph751xBF_IJVW9O2o4vK8yqIxxR3rTW4/s400/Screen+Shot+2012-12-23+at+1.39.05+PM.png" width="400" /></a></div>
<br />
<b>Virtual Machine Network Adapter</b><br />
<br />
I prefer to keep the virtual machine as its own entity on the network. This clears up any issues related to NATing ports and gives our environment the most realistic production feel. <br />
<br />
<br />
<br />
<br />
<br />
<h3>
</h3>
<h3>
</h3>
<div>
<br />
<br />
<br />
<br /></div>
<div>
<br /></div>
<h4>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFDM6ipxbauFPJuXOx4zw6tE1rC8VGUxhhGlVH7mR9ONOFOtoCjNlS1GsP1Gu8TU97aJkCMruMczQsyOyl_DVpt7TRZni91JgJlXsqioF4XdO0L0PIcvKpo-XvCirh1Wm9nKvtOcw4rXE/s1600/desktop-1210-install-welcom.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFDM6ipxbauFPJuXOx4zw6tE1rC8VGUxhhGlVH7mR9ONOFOtoCjNlS1GsP1Gu8TU97aJkCMruMczQsyOyl_DVpt7TRZni91JgJlXsqioF4XdO0L0PIcvKpo-XvCirh1Wm9nKvtOcw4rXE/s320/desktop-1210-install-welcom.jpg" width="320" /></a></h4>
<h3>
Install Ubuntu OS From Media</h3>
<div>
Boot up the virtual machine and Install ubuntu.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj26OZuUVGYKGA-DXxwL1a4BuQZ0o6MhGhTHxmZAZ2E76pafvqMkr2qd5FOne2vDSOI8nqVn1nS4QcOnDzRMKGuuO5Dqfw1lPAFcLI4JapUKKDfsPjRpZgiCCXBLindE3mqp7fortK8ApQ/s1600/desktop-1210-install-prepar.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj26OZuUVGYKGA-DXxwL1a4BuQZ0o6MhGhTHxmZAZ2E76pafvqMkr2qd5FOne2vDSOI8nqVn1nS4QcOnDzRMKGuuO5Dqfw1lPAFcLI4JapUKKDfsPjRpZgiCCXBLindE3mqp7fortK8ApQ/s1600/desktop-1210-install-prepar.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a></div>
<br />
<h3>
<b>
Update OS </b></h3>
I prefer to keep my linux machines patched.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ sudo apt-get update</i></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ sudo apt-get upgrade</i></span><br />
<br />
<h2>
Install Dependencies</h2>
<div>
Before we can really get going with Webstorm, we need to get vmware-tools installed for all that full screen resolution glory. Once installed, we need to pick up g++.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ sudo apt-get install g++</i></span></div>
<div>
<br /></div>
Now that we have our Ubuntu Linux Desktop 13.04 development environment setup, install nodejs, and also install our global node modules/binaries.<br />
<br />
<h4>
<b>Install git, setup bare repository, use /opt for development</b></h4>
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ sudo apt-get install git</span></i><br />
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ sudo mkdir /git</span></i><br />
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ sudo mkdir /opt</span></i><br />
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ sudo chown <username> /opt</span></i></div>
<br />
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ sudo chown <username> /git</span></i><br />
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></i>
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ mkdir /git/myFirstWebServer.git</span></i><br />
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ cd /git/myFirstWebServer.git</span></i><br />
<i><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ git --bare init</span></i><br />
<br />
<b>Install Node v0.10.15 (latest)</b><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;">$ wget <a href="http://nodejs.org/dist/v0.10.15/node-v0.10.15.tar.gz" target="_blank">http://nodejs.org/dist/v0.10.15/node-v0.10.15.tar.gz</a></span><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ tar zxvf </i></span><i style="font-family: Verdana, sans-serif; font-size: small;">node-v0.10.15.tar.gz</i><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ cd </i></span><i style="font-family: Verdana, sans-serif; font-size: small;">node-v0.10.15</i><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ ./configure</i></span><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ make</i></span><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ sudo make install</i></span><br />
<br />
<b>Install global node modules and their binaries</b><br />
<b><br /></b>
<span style="font-family: Verdana, sans-serif; font-size: x-small;">$ sudo npm install -g coffee-script express bower</span><br />
<br />
<h2>
<b>
Creating our basic Express Web Server</b></h2>
<b><br /></b>
I prefer to keep all my custom code inside /opt, so we will setup our Express web server there. Set permissions as necessary. We will also initialize git at this time.<br />
<br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ cd /opt</i></span><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ express -s myFirstWebserver</i></span><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ cd myFirstWebserver</i></span><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ git init</i></span><br />
<span style="font-family: Verdana, sans-serif; font-size: x-small;"><i>$ npm install</i></span><br />
<br />
Great. Now that we have our basic Express Web Server, we can run it with 'node app.js' and visit it by going to <span style="font-family: Verdana, sans-serif; font-size: x-small;"><a href="http://localhost:3000/" target="_blank">http://localhost:3000</a></span> using a web browser. <br />
<br />
Next, we want to convert all of our assets to coffee. Use an existing js2coffee converter, such as <a href="http://js2coffee.org/" target="_blank">http://js2coffee.org</a>. Convert and clean up your js files into coffee.<br />
<br />
Your app.coffee code could look something like this:<br />
<br />
<script src="https://gist.github.com/mikekunze/5778318.js"></script>
<br />
<h2>
Configuring bower and components</h2>
We will use bower to install client libraries. Its like npm, and helps streamline deployments. For our particular configuration, we choose the project folder's pubilc directory to store components since its already routed through express.<br />
<b>Configure bower </b><span style="font-size: x-small;"><a href="https://github.com/bower/bower#configuration" target="_blank">(docs)</a></span><br />
<b><br /></b><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">
edit /home/<username>/.bowerrc:</span><br />
<br />
<script src="https://gist.github.com/mikekunze/5778363.js"></script>
<br />
<br />
Now that we have our basic rc setup for bower, we need to specify our client library dependencies in our project.<br />
<br />
edit /opt/myFirstWebsite/component.json<br />
<br />
<script src="https://gist.github.com/mikekunze/5778390.js"></script>
Bower is completely setup and ready to install packages.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ cd /opt/myFirstWebsite</i></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ bower install</i></span><br />
<br />
Now that we have our client libraries, lets add references to them in the website's layout.<br />
edit ./views/layout.jade:<br />
<br />
<script src="https://gist.github.com/mikekunze/5778432.js"></script>
<br />
<br />
<br />
<br />
<h2>
Commit the changes and push to the code repository</h2>
<div>
Before we commit our changes, we need to setup a .gitignore file so we dont blast the code repository with stuff bower and npm deal with. </div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">edit /opt/myFirstWebsite/.gitignore</span></div>
<div>
<script src="https://gist.github.com/mikekunze/5778470.js"></script>
<br />
<div>
<br /></div>
<div>
Now that we have successfully created a starter bootstrap enabled website, its time to commit our changes and push them to our source repository. First, we will setup the repository origin.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ cd /opt/myFirstWebsite </i></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ git remote add origin /git/myFirstWebsite.git</i></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i><br /></i></span>
Next, we will commit the changes to origin: </div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ git add .</i></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ git commit -a</i></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><i>$ git push -u origin master</i></span></div>
<br />
<br /></div>
Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-71703669996194527182013-02-10T15:51:00.001-08:002013-02-10T15:51:58.555-08:00Node E-Mail Contact Form Submission Rate Limiting With SessionsUsing mongoose and coffee-script, I will layout a simple approach to limiting POST queries by client session id.<br />
<br />
There are specific requirements for this POST route:<br />
<br />
<ul>
<li>E-mail uses Gmail service account (provided by Node-Mailer)</li>
<li>Provide user feedback in real time (res.send)</li>
<li>Rate limit user's ability to POST over a defined interval (timestamps)</li>
</ul>
<br />
<br />
Lets create our model. This collection will store our users submissions by session id.<br />
<br />
<br />
<script src="https://gist.github.com/mikekunze/4751516.js"></script>
Now lets define our app.post function.<br />
<br />
<script src="https://gist.github.com/mikekunze/4751509.js"></script>
Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-72248578067057657822012-09-16T09:02:00.001-07:002012-09-17T19:37:34.660-07:00MicroServer ESXi WhiteBox Part 3 - VM Backup GUI with ghettoVCB.jsHello World. Ive spent the last few weeks working on a GPL interface for <a href="http://communities.vmware.com/docs/DOC-8760">ghettoVCB</a> called ghettoVCB.js. This interface uses NodeJS (Express + MongoDB) + ExtJS 4.1 for its main components. This will probably be the last ExtJS project I do for a while. I think time is now for me to begin learning objective-c for OSX/iOS apps. That, and I will focus on Twitter's Bootstrap and jQuery for a more light weight set of widgets than what ExtJS offers me. I also shouldn't settle in with a single product ie ExtJS. Sencha. Interacting with the company has a lot to be desired, but enough with that.<br />
<br />
<h2>
ghettoVCB.js</h2>
A little history, first. This project was inspired by a bioinformatics pipeline workflow web app I wrote called scriptQueue that remotely ran fastQC jobs on a compute node through SSH, and pulled the HTML results with SSHFS and display them through the web service. The goal of that project was to allow for additional workflow types and the workflows themselves were to be absolutely and completely flexible.<br />
<br />
scriptQueue used the child_process module that comes with NodeJS. With the spawning of a child SSH process, I realized through the events of watching its standard output and process status (running/not running), I was able to keep complete control of not only any arbitrary command line application on the host OS, but on any remote host through SSH piping. Awesome!<br />
<br />
Back to ghettVCB.js, which is the topic of this post. Naturally, one of the workflows that came to mind for scriptQueue was a <a href="http://communities.vmware.com/docs/DOC-8760">ghettoVCB</a> workflow for my MicroServer ESXi WhiteBox. Since scriptQueue was designed with the workflow concept (both server side and client side) forking the project was as easy as adding a new workflow form client side and workflow route server side. Bam.<br />
<br />
For the <a href="http://communities.vmware.com/docs/DOC-8760">ghettoVCB</a> workflow, the webapp creates a folder './ghettoVCB' in the apps running path. It then uses SSHFS to mount the remote ESXi script path which contains ghettoVCB.sh. When the "Submit ghettoVCB" button is clicked, the webapp will try and save a configuration file (ghettoVCB.js.conf) and a list of virtual machines to backup (vms_to_backup.conf) on the remote ESXi host through the ghettoVCB mount.<br />
<br />
Once the files are saved, the web app will send off a remote child process with the proper command line parameters pointing to ghettoVCB.sh script, the configuration and list of vm files.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU-g47G8tHqr5P2y4rWpflpD_ye95JiRcYqEOHir4gghCbDDyBRo0mGUi1WxgsV4bmiVICBTBB5tyinnysrbxBAjtdR4iNzvs08HAGidC_nxDTivWaPr6mmZOToaJER9ADZ1tC4mzrN-E/s1600/Screen+Shot+2012-09-16+at+10.21.38+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="565" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU-g47G8tHqr5P2y4rWpflpD_ye95JiRcYqEOHir4gghCbDDyBRo0mGUi1WxgsV4bmiVICBTBB5tyinnysrbxBAjtdR4iNzvs08HAGidC_nxDTivWaPr6mmZOToaJER9ADZ1tC4mzrN-E/s640/Screen+Shot+2012-09-16+at+10.21.38+AM.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
Below is a view of the History Grid, which visualizes all the meta-data stored for a child process. This is where mongoDB comes in. I store all standard output and standard error results in the meta-data of the child process in mongoDB through the mongoose API. I also keep tabs on the PID number, its status, start and end dates.</div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
As you can see, the display is a little interesting when we get to the Clone percentage. The normal stdout replaces the same line with an updated number, but the way I currently save output, it appends a new line. No biggy.</div>
<div style="text-align: start;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: start;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYZExAibktYgr4zWtqUID7fb2w3YwMfbqyUWyGWh4AEVgjtUKq7CtB3bkzTLLQflr2SnAohpf6AY2ixhaVs7IqovFeMs9Lo5pzoSM1fUJJOC7n1uE1fQ04dqtRplgcRqc_wRwKqzb77eQ/s1600/Screen+Shot+2012-09-16+at+10.27.33+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYZExAibktYgr4zWtqUID7fb2w3YwMfbqyUWyGWh4AEVgjtUKq7CtB3bkzTLLQflr2SnAohpf6AY2ixhaVs7IqovFeMs9Lo5pzoSM1fUJJOC7n1uE1fQ04dqtRplgcRqc_wRwKqzb77eQ/s640/Screen+Shot+2012-09-16+at+10.27.33+AM.png" width="640" /></a></div>
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
So there you have it. That's ghettoVCB.js, a GPL ExtJS 4.1 / NodeJS WebApp GUI for running ghettoVCB.sh on a remote ESXi host.</div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<a href="https://github.com/mikekunze/ghettoVCB.js">https://github.com/mikekunze/ghettoVCB.js</a></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
<div style="text-align: start;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com2tag:blogger.com,1999:blog-5361461834637096019.post-27022379888487568832012-09-10T16:57:00.000-07:002012-09-10T16:57:49.512-07:00MicroServer ESXi WhiteBox Part 2 - VM Backups easily with ghettoVCB<h4>
</h4>
Hello world. Part two of my MicroServer ESXi WhiteBox series will be discussing how to back up those precious virtual machines made with our free ESXi server. Fortunately for us, the community has provided a very powerful ghettoVCB script to do just that.<br />
<div>
<br /></div>
<div>
You can find the documentation <a href="http://communities.vmware.com/docs/DOC-8760">here</a>. </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhE7SHLg5el9_KHn0T-8aMMKKPBb3bifi-q96SYDODAr4eZvK9zMEEBSxEj1QtmJBy0i-CK5r9nyydn1hjJCeT2BWtU-nyOB9i7NKuaNGOsglfePUos9EI7jKGZYCVfq_tuTomtXVDT4gc/s1600/Screen+Shot+2012-09-10+at+6.12.27+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhE7SHLg5el9_KHn0T-8aMMKKPBb3bifi-q96SYDODAr4eZvK9zMEEBSxEj1QtmJBy0i-CK5r9nyydn1hjJCeT2BWtU-nyOB9i7NKuaNGOsglfePUos9EI7jKGZYCVfq_tuTomtXVDT4gc/s1600/Screen+Shot+2012-09-10+at+6.12.27+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhE7SHLg5el9_KHn0T-8aMMKKPBb3bifi-q96SYDODAr4eZvK9zMEEBSxEj1QtmJBy0i-CK5r9nyydn1hjJCeT2BWtU-nyOB9i7NKuaNGOsglfePUos9EI7jKGZYCVfq_tuTomtXVDT4gc/s640/Screen+Shot+2012-09-10+at+6.12.27+PM.png" width="640" /></a></div>
<div>
<br />
<br />
<h4>
<b>Installation</b></h4>
<hr />
<br />
First, make sure you can SSH to the ESXi box. <a href="http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vcli.migration.doc_50%2Fcos_upgrade_technote.1.4.html">Enable remote console via the vSphere client</a>.<br />
<br />
You can download the <a href="https://github.com/lamw/ghettoVCB/tarball/master">tar.gz here</a>. Extract the ghettoVCB folder to a datastore using the vSphere client's datastore Browser inside a scripts folder. <br />
<br />
SSH to your ESXi box. Make sure the script itself is executable (<span style="font-size: x-small;">ghettoVCB.sh</span>), then edit the configuration file (<span style="font-size: x-small;">ghettoVCB.conf</span>). Finally, you will need to create a list of virtual machines. I keep this part simple (for more options, like targeting specific disks, <a href="http://communities.vmware.com/docs/DOC-8760">see the docs</a>). I create a file called vms_to_backup and per line, include the names of the virtual machines I intend to backup and maintain two copies.<br />
<br />
<span style="font-size: x-small;"># cd /vmfs/volumes/2TB_Spindle/scripts/ghettoVCB</span><br />
<span style="font-size: x-small;"># chmod 755 ghettoVCB.sh</span><br />
<span style="font-size: x-small;"># vi ghettoVCB.conf</span><br />
<span style="font-size: x-small;"><script src="https://gist.github.com/3694867.js?file=ghettoVCB.conf"></script></span><br />
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;"># vi vms_to_backup</span><br />
<span style="font-size: x-small;"><script src="https://gist.github.com/3694872.js?file=vms_to_backup"></script></span><br />
<br />
<br />
Good enough for my backup sanity. Lets give the script a dryrun to see if we have any problems.<br />
<br />
<span style="font-size: x-small;"># ./ghettoVCB.sh -f vms_to_backup -g ./ghettoVCB.conf -d dryrun</span><br />
<span style="font-size: x-small;"></span><br />
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;"><b>2012-09-09 17:54:15 -- info: ###### Final status: OK, only a dryrun. ######</b></span><br />
<div>
<span style="font-size: x-small;"><br /></span></div>
<br />
<br />
<br />
<br />
If all looks good, lets give it a whirl.<br />
<div>
<br /></div>
<br />
<span style="font-size: x-small;"># </span><span style="font-size: x-small;">./ghettoVCB.sh -f vms_to_backup -g ./ghettoVCB.conf</span><br />
<span style="font-size: x-small;"><br /></span>
<br />
<span style="font-size: x-small;"><b>2012-09-09 21:22:55 -- info: Successfully completed backup for vCenter!</b></span><br />
<span style="font-size: x-small;"><b><br /></b></span>
<span style="font-size: x-small;"><b>2012-09-09 21:22:56 -- info: ###### Final status: All VMs backed up OK! ######</b></span><br />
<span style="font-size: x-small;"><b><br /></b></span>
<span style="font-size: x-small;"><b>2012-09-09 21:22:56 -- info: ghettoVCB LOG END </b></span><br />
<div style="font-size: small;">
<br /></div>
<br />
<span style="font-size: x-small;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYEQm3EwPkEq0wnvLrGTTaN5GeRWAGPZqjJy6h5t_w7mhNUcmRApLDZn7tCk5rsEgaBqSWRdkA0JjIznqKkb3EY3RhUMjwxgvYxFYcrGS0ZA7D4FgLxq52rsXIgdhDjRXARPqzYnkMwh0/s1600/Screen+Shot+2012-09-10+at+6.27.11+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="284" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYEQm3EwPkEq0wnvLrGTTaN5GeRWAGPZqjJy6h5t_w7mhNUcmRApLDZn7tCk5rsEgaBqSWRdkA0JjIznqKkb3EY3RhUMjwxgvYxFYcrGS0ZA7D4FgLxq52rsXIgdhDjRXARPqzYnkMwh0/s640/Screen+Shot+2012-09-10+at+6.27.11+PM.png" width="640" /></a></div>
<br />
<br />
The next step is to add a crontab entry and run it at a specific interval of your choosing. Since the conf file says VM_BACKUP_ROTATION_COUNT=2, it will keep two run results of the backup script.<br />
<br />
<br />
Good luck.<br />
</div>
Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-89691578527600881542012-08-15T11:11:00.001-07:002012-09-10T16:59:29.331-07:00MicroServer ESXi WhiteBox Part 1 - The Hardware<span style="font-size: large;"><b>MicroServer ESXi White(silver)Box Part1</b></span><br />
<br />
I recently researched a dedicated home office <a href="http://www.vmware.com/products/vsphere/esxi-and-esx/overview.html">ESXi</a> 5 server. The goal was to have the capacity to run 4-8 virtual machines. My implementation includes two dedicated windows server domain controllers, two linux servers for a continuously integrated development environment, and a dedicated storage appliance (running ubuntu linux) to host fast SSD and large spindle disk storage. This "datacenter in a box" was to have the capacity to add disks, passthrough PCIe devices, and have the potential to team with more (with a central NAS) for a complete "cloud" environment on the cheap. I wanted this datacenter in a box to simply "plug n' play". No dependency on keyboard/mouse or display. Everything would exist virtually via the network and thus power and ethernet are its only requirements. Ultimately this box would end up hidden somewhere in a closet or under a desk, out of sight. The greatest requirement was to be small, quiet and use as little power as possible. <br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihHICqw8LU3jtvpfM3dLzuCB-baDqr_4nTiwWE93gfrRRS5Qck-eyvvwBzVrm5L_Ozr6yqBYH4vBpdK2nYmwrXdCsB4uRanRX5bJPYbTqgrN7KzKW1wCSPMUT5L1aK388VrtiCtJPr5Do/s1600/IMG_1266.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihHICqw8LU3jtvpfM3dLzuCB-baDqr_4nTiwWE93gfrRRS5Qck-eyvvwBzVrm5L_Ozr6yqBYH4vBpdK2nYmwrXdCsB4uRanRX5bJPYbTqgrN7KzKW1wCSPMUT5L1aK388VrtiCtJPr5Do/s200/IMG_1266.jpg" width="148" /></a>After a bit of research on the concept of creating one of these home server whiteboxes (<a href="http://ultimatewhitebox.com/">http://ultimatewhitebox.com/</a>), I found a nice blog entry called The Baby Dragon: <a href="http://rootwyrm.us.to/2011/09/better-than-ever-its-the-babydragon-ii/">http://rootwyrm.us.to/2011/09/better-than-ever-its-the-babydragon-ii/</a>, which detailed a fairly recent (as of 2011) configuration for a home office whitebox ESXi server. From there I compiled an order on newegg.<br />
<br />
Part 1 of this series will detail the hardware procurement and installation. Later parts will discuss ESXi installation, maintenance and caveats.<br />
<br />
<br />
<br />
<br />
<span style="font-size: large;"><b>Motherboard, Processor, and Ram</b></span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEiVQ5ZX59spqm15P3unzP_83mpwuckmh8z7KM_W26FHpuDhOpJbguYGs4KHI8ql_oVZPfqKg7KznLQXTmox4Ps9eFHTv_6XxV1j8ukrdS0PxDdjZilksqsPkMNuQosjXpcwHBBR8Z7FU/s1600/IMG_1260.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEiVQ5ZX59spqm15P3unzP_83mpwuckmh8z7KM_W26FHpuDhOpJbguYGs4KHI8ql_oVZPfqKg7KznLQXTmox4Ps9eFHTv_6XxV1j8ukrdS0PxDdjZilksqsPkMNuQosjXpcwHBBR8Z7FU/s400/IMG_1260.jpg" width="400" /></a></div>
The most critical aspect in terms of ESXi 5 is to have as many enterprise features as possible for extendability. Virtualization technologies including hardware passthrough is one of the most important. With passthrough, we can pass PCI express or USB devices directly to a virtual machine. That means I could add a RAID card, drop in a few disks, and add it to a linux storage virtual machine, or add a TV tuner card to a Windows 7 Media Center Server.<br />
<br />
One of the most popular server boards at newegg right now is from Supermicro, the <a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253">X9SCM-F-O</a>. This MicroATX motherboard supports VT-d (via the Xeon processor), 32GB of RAM, and IPMI to name a few. Like the Baby Dragon II, I will be using the <a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16819117286">Xeon 1230</a> processor. For ram, I chose <a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16820239135">Kingston's 8GB Hynix</a> RAM which worked out to be a great deal. I only bought 16GB so far, since I am far from maximizing use of that much RAM in my setup, and I am currently encountering overheating issues with the CPU's stock heatsink and fan which is preventing me from adding more VMs. Overheating was occurring when I was doing massive Windows Updates for two Server 2008 installs that were running on the fast SSD datastore. Apparently the SSD is so fast that the processor had a hard time keeping up without getting too hot.<br />
<br />
<i><span style="font-size: x-small;">I am not entirely sure I have a real overheating issue, because supermicro dumbs down the IPMI interface to the temperature sensor on the CPU giving you OK, WARN, and CRITICAL values instead of the actual temperature. Not very useful when you are trying to see if CRITICAL means its reached Intel's recommendation for maximum temperature or not. Currently, this is my biggest issue with the setup. When CRITICAL is reached, the motherboard beeps very annoyingly until you lessen its load. According to intel's <a href="http://ark.intel.com/products/52271?wapkw=xeon+e3-1230">specs</a>, the maximum operating temperature is 69 degrees c, which I could be reaching with the stock heatsink / fan.</span></i><br />
<br />
<br />
<span style="font-size: large;"><b>Chassis and Power Supply</b></span><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgs8u_NaPLji5TPb84E1lFW5SLo91MhRJmQ4aYRoTodZe2LwpU6TowKNQrk9GzVZXeC2_DRRAt9WBq7JwVeK3UBed-ysETPRMrlJRa7YtnbYavL9QAizWR3TjGe4B3gFdP9SfiW2FI06UY/s1600/IMG_1262.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgs8u_NaPLji5TPb84E1lFW5SLo91MhRJmQ4aYRoTodZe2LwpU6TowKNQrk9GzVZXeC2_DRRAt9WBq7JwVeK3UBed-ysETPRMrlJRa7YtnbYavL9QAizWR3TjGe4B3gFdP9SfiW2FI06UY/s400/IMG_1262.JPG" width="400" /></a></div>
<b>Chassis</b><br />
For chassis I chose the <a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16811112299">Lian-Li v354A</a> MicroATX Desktop Case. The case supports up to seven 3.5 inch drives and two 2.5 drives. The image to the left shows a 2TB spindle disk on top and a 240GB SSD on the bottom of the drive cage. The lower drive cage has been removed.<br />
<br />
<b>Power Supply</b><br />
For power supply, I chose the <a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16817151088">SeaSonic X650</a> Gold EPS 12V. It is modular and rarely runs its fan.<br />
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b>ESXi Installation</b></span><br />
For my build, as well as for the Baby Dragon II, a USB thumb drive was used for the ESXi Operating System. Since the motherboard has an onboard USB port, it seemed entirely logical. Its slightly faster than a spindle disk and definitely takes up less space.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_w-6WIGCrzy8RLr4X4Piyw8vzHEUCR92wRqU7jFdMfKWSYHyvyTxOweVzHCF9Xj0snKBAOfJIDM2UaUM8YGzHN20fmHrCjivb8sERVOtmH-Gjh4mMcOT9km51PEEJ1_fEp0yMgpedDRU/s1600/IMG_1259.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_w-6WIGCrzy8RLr4X4Piyw8vzHEUCR92wRqU7jFdMfKWSYHyvyTxOweVzHCF9Xj0snKBAOfJIDM2UaUM8YGzHN20fmHrCjivb8sERVOtmH-Gjh4mMcOT9km51PEEJ1_fEp0yMgpedDRU/s400/IMG_1259.jpg" width="400" /></a></div>
I used VMware fusion to install ESXi 5 on my thumb drive. VMware Player will work as well. Pass through the USB thumb drive to a new blank virtual machine. Do not worry about adding a harddrive to this VM, as you wont need it. Boot up the blank VM with the ESXi installer iso and follow the prompts, choosing the USB thumb drive as the install destination.<br />
<br />
This image shows the thumb drive on the left side. In the middle are the 6 SATA ports (2 SATA3 and 4 SATA2). In the way of the shot is the lower drive cage.<br />
<br />
Once the system boots up with the thumb drive, you can use the management console (via IPMI) to configure networking.<br />
<br />
<span style="font-size: large;"><b><br /></b></span>
<span style="font-size: large;"><b>ESXi Updating without vCenter (or Update Manager)</b></span><br />
Since I am doing the free thing, I will need to fill gaps that vCenter server would fill and patching is one of them. Thankfully, the process is quite easy. First, we need to know what updates are available.<br />
<br />
SSH into your ESXi server and run the following command:<br />
<br />
# esxcli software sources vib list -d http://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep Update<br />
<br />
A list of updates (if you have any) will return. Pick up the latest cumulative update at the vmware website: <a href="http://www.vmware.com/patchmgr/download.portal">http://www.vmware.com/patchmgr/download.portal</a>, and upload it to a datastore on the ESXi host.<br />
<br />
# esxcli software vib update -d /vmfs/volumes/<b>datastore_name</b>/<b>patch_location</b><br />
<br />
In another console session, launch the following to keep an eye on the status:<br />
<br />
# watch tail -n 50 /var/log/esxupdate.log<br />
<b><br /></b>
<b>Issues with updating?</b><br />
<span style="font-size: x-small;">If you have any issues with installing the updates, and its related to "altbootbank", then you will need to "fix" a few partitions. </span><br />
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;"># dosfsck -v -r /dev/disks/<b>partition_id</b><disk id="id" partition="partition"></disk></span><br />
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;">My disk partition id on the thumb drive that was bad is called "mpx.vmhba32:C0:T0:L0:5". To fix it, I said Y at the prompt when it notified me of errors.</span><br />
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-78698876685655286552012-07-26T16:29:00.000-07:002012-07-26T17:04:34.380-07:00Creating SharePoint 2010 reports with Javascript and Twitter BootstrapSharePoint 2010 is a decent CMS out there from Microsoft. Great for customized document storage, customized data lists, and of course Office Web Apps for cross-browser web based viewers/editors (word, excel, powerpoint, visio, etc). SharePoint 2010 can also be extended to open source technologies rather easily with its REST services in order to create customized HTML reports rather than viewing through the clunky ribbon interface in SharePoint itself.<br />
<br />
In this presentation, I will use <a href="http://nodejs.org/">NodeJS</a>, <a href="http://expressjs.com/">Express</a>, <a href="http://jade-lang.com/">Jade</a>, <a href="http://coffeescript.org/">coffee-script</a>, and <a href="http://twitter.github.com/bootstrap">Twitter's Bootstrap</a> to construct a simple list report generator that page breaks on every item. We will pretend that the SharePoint 2010 data is a custom list with three required columns and two optional columns that may not contain data: Title (string), Content (rich text), and Feedback (rich text), TextArea1, TextArea2. The last two are obviously the optional columns.<br />
<br />
Imagine that this custom list contains a lot of text in each column. Viewing this in SharePoint would show it as a grid, with each associated property listed horizontally (think spreadsheet). There would be much scrolling or resizing of the browser. Why cant we view this data like a book or a white paper? That is what I intend to accomplish in this presentation.<br />
<br />
First of all, you'll need to pick up all the tools. I am using Node 0.8.3 with the following modules: Express (3.0.0beta7), Jade (0.27.0), coffee-script (1.3.3), and request (2.9.203). All of the modules are on npm. See <a href="http://portalstack.blogspot.com/2012/07/installing-coffee-script-cli.html">here</a> for instructions on running coffee-script from the command line.<br />
<br />
Next, pick up the latest bootstrap and its <a href="http://twitter.github.com/bootstrap/javascript.html">javascript plugins</a>. Finally go get <a href="http://jquery.com/">jQuery</a>.<br />
<br />
<br />
$ mkdir reportGenerator<br />
$ cd reportGenerator<br />
$ mkdir public routes views<br />
<br />
<br />
<br />
Now that we have our public folder, lets put the client side stuff in there.<br />
<br />
$ mv ~/Downloads/bootstrap ./public<br />
$ mv ~/Downloads/jQuery.min.js ./public/bootstrap/js<br />
<br />
<br />
Lets setup our web server.<br />
<br />
$ vim app.coffee<br />
<script src="https://gist.github.com/3185121.js?file=app.coffee">
</script><br />
<br />
<br />
<br />
Now we need to create the route app.coffee depends on.<br />
<br />
$ vim ./routes/index.coffee<br />
<script src="https://gist.github.com/3185143.js?file=index.coffee">
</script><br />
<br />
<br />
Cool. So this route renders a Jade template. Lets make the layout and index templates.<br />
<br />
$ vim ./views/layout.jade<br />
<script src="https://gist.github.com/3185152.js?file=layout.jade">
</script><br />
<br />
$ vim ./views/index.jade<br />
<script src="https://gist.github.com/3185185.js?file=index.jade">
</script><br />
<br />
<br />
<br />
Now that the templates are done, we are ready to rock. Start it up<br />
<br />
$ coffee app.coffee<br />
<br />
Print to PDF. Look at that! A nice report generated with page breaks on each main topic (item) based on SharePoint 2010 data in a simple yet expressive way.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-55464167099107750092012-07-21T08:46:00.000-07:002012-07-21T08:46:43.521-07:00Installing coffee-script cli<br />
Today I will be updating my box to the latest nodejs and after doing so, installing the coffee-script command line interface which allows for coffee-script to be run on such things as crontab, init.d, or even nagios (<a href="http://portalstack.blogspot.com/2012/07/sending-custom-nagios-notification.html">see other post</a>). I will also be installing nodemon which will act as a daemon for my coffee-script web applications. Nodemon will watch for changes and restart the application if it finds any, which is pretty neat.<br />
<br />
$ wget http://nodejs.org/dist/v0.8.3/node-v0.8.3.tar.gz<br />
$ tar zxvf node-v0.8.3.tar.gz<br />
<br />
$ cd node-v0.8.3<br />
<br />
<br />
$ ./configure<br />
<br />
<i><span style="font-size: x-small;">{ 'target_defaults': { 'cflags': [],</span></i><br />
<i><span style="font-size: x-small;"> 'default_configuration': 'Release',</span></i><br />
<i><span style="font-size: x-small;"> 'defines': [],</span></i><br />
<i><span style="font-size: x-small;"> 'include_dirs': [],</span></i><br />
<i><span style="font-size: x-small;"> 'libraries': []},</span></i><br />
<i><span style="font-size: x-small;"> 'variables': { 'host_arch': 'x64',</span></i><br />
<i><span style="font-size: x-small;"> 'node_install_npm': 'true',</span></i><br />
<i><span style="font-size: x-small;"> 'node_install_waf': 'true',</span></i><br />
<i><span style="font-size: x-small;"> 'node_prefix': '',</span></i><br />
<i><span style="font-size: x-small;"> 'node_shared_openssl': 'false',</span></i><br />
<i><span style="font-size: x-small;"> 'node_shared_v8': 'false',</span></i><br />
<i><span style="font-size: x-small;"> 'node_shared_zlib': 'false',</span></i><br />
<i><span style="font-size: x-small;"> 'node_use_dtrace': 'false',</span></i><br />
<i><span style="font-size: x-small;"> 'node_use_etw': 'false',</span></i><br />
<i><span style="font-size: x-small;"> 'node_use_openssl': 'true',</span></i><br />
<i><span style="font-size: x-small;"> 'target_arch': 'x64',</span></i><br />
<i><span style="font-size: x-small;"> 'v8_no_strict_aliasing': 1,</span></i><br />
<i><span style="font-size: x-small;"> 'v8_use_snapshot': 'true'}}</span></i><br />
<i><span style="font-size: x-small;">creating ./config.gypi</span></i><br />
<i><span style="font-size: x-small;">creating ./config.mk</span></i><br />
<i><span style="font-size: x-small;"><br /></span></i><br />
<br />
$ make<br />
$ sudo make install<br />
$ sudo npm install -g coffee-script<br />
$ sudo npm install -g nodemon<br />
<br />
<br />
<br />
<i><span style="font-size: x-small;"><br /></span></i><br />
<i><span style="font-size: x-small;"><br /></span></i><br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-74262383315230630222012-07-14T09:05:00.000-07:002013-08-19T14:47:55.631-07:00Sending custom Nagios notification emails with coffeescriptLets imagine for a second that you are using Nagios to monitor systems and services. Nagios can send you notifications upon events, but the default email notification command is kind of boring.<br />
<div>
<br /></div>
<div>
Lets spice it up with coffeescript!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<h2>
<b>Nagios</b><hr />
</h2>
</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">define contact{</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> contact_name mike</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> alias Mike Kunze</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> service_notification_period 24x7</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> host_notification_period 24x7</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> service_notification_options w,u,c,r</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> host_notification_options d,r</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> service_notification_commands <b>notify-service-with-nodejs</b></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> host_notification_commands notify-host-by-email</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> email <your address="" email=""></your></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">}</span></div>
</div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"></span><br />
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">define command {</span></div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">
</span>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> command_name notify-service-with-nodejs</span></div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">
<div>
command_line <b>/opt/bin/notify-service.coffee</b> "$HOSTNAME$" "$SERVICEDESC$" "$HOSTADDRESS$" "$NOTIFICATIONTYPE$" "$SERVICESTATE$" "$LONGDATETIME$" "$SERVICEOUTPUT$"</div>
<div>
}</div>
</span></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<h2>
CoffeeScript<hr />
</h2>
<br /></div>
<div>
<br /></div>
<div>
This script will be executed by nagios and will contain the macros provided by the nagios command.</div>
<div>
<br />
<script src="https://gist.github.com/mikekunze/3111907.js"></script>
</div>Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-67905638825942353732012-06-16T08:46:00.001-07:002012-06-16T08:46:39.608-07:00Windows 7 Updates (service not running error)Today, I brought back up an old machine of mine running windows 7. It had been a good 6 months since I've last used it, so naturally, windows updates was not working.<br />
<br />
I managed to find a nice thread on social.technet: <a href="http://social.technet.microsoft.com/Forums/en/w7itprogeneral/thread/6a8889a8-65b2-4012-9cf8-2689f47b21e4">http://social.technet.microsoft.com/Forums/en/w7itprogeneral/thread/6a8889a8-65b2-4012-9cf8-2689f47b21e4</a><br />
<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;">stop Windows Update Service</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
<span style="font-family: 'Courier New', Courier, monospace;">delete</span><span style="font-family: 'Courier New', Courier, monospace;"> C:\windows\softwaredistribution\*.*</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
<span style="font-family: 'Courier New', Courier, monospace;">start Windows Update Service</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
<span style="font-family: 'Courier New', Courier, monospace;">check for updates</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
Hope this helps if you run into the same issue.Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-6519364990250141652012-06-11T16:47:00.001-07:002012-06-11T16:50:38.522-07:00NodeJS and OpenShift, a PortalGNU presentation<div class="separator" style="clear: both; text-align: left;">
Today, I am going to get my platform service configured and ready for development. For PortalGNU, I will be using Red Hat OpenShift. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
First we select a web cartridge. In my case, I will select Node.JS.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjktr5y8BCHB2_jc_COcC5AThD-kqrjwGR5njdvSZ65-GWaSBgB8fDuokWp3oZ2F1kBuGhWTOeNBknDS0OQkChcyqMMza5iBZQCjUQiDQnnQGLBzK16rXCQ0mYfoWjb1kJ1oaIQNqPLwXI/s1600/Screen+Shot+2012-06-11+at+6.13.33+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="315" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjktr5y8BCHB2_jc_COcC5AThD-kqrjwGR5njdvSZ65-GWaSBgB8fDuokWp3oZ2F1kBuGhWTOeNBknDS0OQkChcyqMMza5iBZQCjUQiDQnnQGLBzK16rXCQ0mYfoWjb1kJ1oaIQNqPLwXI/s640/Screen+Shot+2012-06-11+at+6.13.33+PM.png" width="640" /></a></div>
<br />
<br />
Give the application a name.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaEVSqAAIHDTeXFZRz-aIur_QPdALKxDo9SUXRnKVdN9v8bKeyMJUQ_FNQlwpuPEcUoYQLDO7eUjYBZyMOhkn-XuEnG3qcqVhReJSi4GU9yIPJTjXxy4VZjjbwi1cuDp4g9Qy00enGfqs/s1600/Screen+Shot+2012-06-11+at+6.14.08+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaEVSqAAIHDTeXFZRz-aIur_QPdALKxDo9SUXRnKVdN9v8bKeyMJUQ_FNQlwpuPEcUoYQLDO7eUjYBZyMOhkn-XuEnG3qcqVhReJSi4GU9yIPJTjXxy4VZjjbwi1cuDp4g9Qy00enGfqs/s640/Screen+Shot+2012-06-11+at+6.14.08+PM.png" width="640" /></a></div>
<br />
The next screen will include necessary information for setting up git and pub keys. I installed the openshift toolkit (rhc) so I will continue there.<br />
<br />
OpenShift includes shared databases and other features, called Cartridges. In my case, I want a MongoDB cartridge.<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;">$ rhc app cartridge add -a portal -c mongodb-2.0</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br /></span><br />
<br />
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
RESULT:</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
MongoDB 2.0 database added. Please make note of these credentials:</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
Root User: root</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
Root Password: *******</div>
<br />
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
Database Name: portal</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
Connection URL: mongodb://127.0.0.1:27017</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
You can manage your new MongoDB by also embedding rockmongo-1.1</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<span style="font-family: Times; font-size: small;">So at this point, I have mongoDB installed. Let's check out rockmongo, too.</span></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<span style="font-family: Times; font-size: small;"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<span style="font-family: Times; font-size: small;"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<span style="font-size: small;">$ rhc app cartridge add -a portal -c rockmongo-1.1</span></div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: small;">
<span style="font-size: small;"><br /></span></div>
<br />
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;">RESULT:</span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;">rockmongo-1.1 added. Please make note of these MongoDB credentials again:</span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"> RockMongo User : root</span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"> RockMongo Password: *******</span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;">URL: https://portal-portalgnu.rhcloud.com/rockmongo/</span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"><br /></span></div>
Sweet, a web interface. Now I will create a new database for the Portal CMS.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgitIZxANYDm4NW30C8PnXDfroxJwhbuJST6QJR31eX2W8DiVOX4fNHyfkY4AMGoG0lfV-heFRyYsDXvCP8_bW7OLZ_k-luo-UEio_cDwHto_q7crMttZePFL0tuN4iqM_qkEdv_1qNzJs/s1600/Screen+Shot+2012-06-11+at+6.44.39+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="177" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgitIZxANYDm4NW30C8PnXDfroxJwhbuJST6QJR31eX2W8DiVOX4fNHyfkY4AMGoG0lfV-heFRyYsDXvCP8_bW7OLZ_k-luo-UEio_cDwHto_q7crMttZePFL0tuN4iqM_qkEdv_1qNzJs/s400/Screen+Shot+2012-06-11+at+6.44.39+PM.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeLslrACmGGiM06X-uBh62tqtx1M4i9PK6gbR9WXhoPRSfnhoMTLwVlHEF3anMrxTAi6e9Qcafjxox7d7rHML5QGzEGPMFeETZ5TO1aezQsBNxryOkj3DztH-YiYqDY6YSwC6rpe4zyZ0/s1600/Screen+Shot+2012-06-11+at+6.45.42+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="272" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeLslrACmGGiM06X-uBh62tqtx1M4i9PK6gbR9WXhoPRSfnhoMTLwVlHEF3anMrxTAi6e9Qcafjxox7d7rHML5QGzEGPMFeETZ5TO1aezQsBNxryOkj3DztH-YiYqDY6YSwC6rpe4zyZ0/s640/Screen+Shot+2012-06-11+at+6.45.42+PM.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Cool. From now on, package.json in my git repository will dictate how this application works. For now, I have a cool default landing page:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0vSUGu-J8UXaMNjZoKP_lfIN7id5pbJOJa4Ohz0Raal2y9_p-bqJ1r2RbfTJMRL8ba9aN3756irmdDCR-FS8cM35HAmMIcaarH9aQyY0gjcZGh2sV9VEV7BLiGZnpKIT0PrOeE8wrbcQ/s1600/Screen+Shot+2012-06-11+at+6.46.58+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="499" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0vSUGu-J8UXaMNjZoKP_lfIN7id5pbJOJa4Ohz0Raal2y9_p-bqJ1r2RbfTJMRL8ba9aN3756irmdDCR-FS8cM35HAmMIcaarH9aQyY0gjcZGh2sV9VEV7BLiGZnpKIT0PrOeE8wrbcQ/s640/Screen+Shot+2012-06-11+at+6.46.58+PM.png" width="640" /></a></div>
<br />
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-size: x-small;"><br /></span></div>
<br />
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-28747901534743398372012-03-22T14:00:00.001-07:002012-03-22T14:00:14.021-07:00Creating a content driven continuously tested and integrated source controlled website with NodeJS and ExpressCreating a content driven, continuously tested and integrated source controlled website with NodeJS and Express.<br />
<br />
The next set of blog entries will be dedicated to bringing online my pet project PortalGNU.com/org<br />
<br />
Portal GNU will contain content dedicated to the open source movement. The website will act as a blueprint for spawning your own personal NodeJS + Express + MongoDB driven website.<br />
<br />
Before PortalGNU goes live, my blog will be dedicated to describing the process from github to travis-ci to amazon EC2.<br />
<br />
Right now this is a work in progress. More to come.Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-28742274054209196202012-03-16T13:25:00.001-07:002012-03-16T13:30:04.171-07:00Server 8 Beta - Installing Core, Install GUI laterI'm beta testing server 8, which is in public beta (link)<br />
<br />
This is the first time I've tried installing just the core, but it seems to make sense now.<br />
<br />
<br />
<br />
It is an extremely quick way to get a box online and exposing RDP without the hassle of navigating a GUI.<br />
<br />
To quickly get your hostname, networking, and RDP configured, simply run <a href="http://technet.microsoft.com/en-us/library/ee441254(v=ws.10).aspx">sysconfig.cmd</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhT6Z8cLiqQjCojqzkbKpomaHtKkATZU5sN_CrUMNQF2P3aBXhgoOD2RqFyjknE_ABxWNHi-4XsS5pBfqzVdjTtEhY_dxc2X-htIc8qnxb_GTXfGZ4emwYuhZZw3RwUPEWwt-OGb9VltxE/s1600/Screen+Shot+2012-03-16+at+3.29.16+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhT6Z8cLiqQjCojqzkbKpomaHtKkATZU5sN_CrUMNQF2P3aBXhgoOD2RqFyjknE_ABxWNHi-4XsS5pBfqzVdjTtEhY_dxc2X-htIc8qnxb_GTXfGZ4emwYuhZZw3RwUPEWwt-OGb9VltxE/s640/Screen+Shot+2012-03-16+at+3.29.16+PM.png" width="640" /></a></div>
<br />
<br />
You may still use the GUI if you'd prefer. The new windows 8 interface plays well with the apple trackpad. I'll warn you though, this will take a while. In order to later install the GUI tools, you simply open up powershell and issue the following:<br />
<br />
<code>
Add-WindowsFeature Server-Gui-Shell
</code><br />
<code><br /></code><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5CMtyjq4mzFxE3VeDqe3NhA2v5NCDWCN4fTGCSS3T-c0pg6HZaANoZThDgHBR6p-DxrHDmK0k0sgmPh6o6G9hJPgrdsr3nJ_4HSIyhSxkF0Tm_sfzTMKyz2Bo_hn78TXPie5PO-Hw7VU/s1600/Screen+Shot+2012-03-16+at+3.20.56+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="60" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5CMtyjq4mzFxE3VeDqe3NhA2v5NCDWCN4fTGCSS3T-c0pg6HZaANoZThDgHBR6p-DxrHDmK0k0sgmPh6o6G9hJPgrdsr3nJ_4HSIyhSxkF0Tm_sfzTMKyz2Bo_hn78TXPie5PO-Hw7VU/s640/Screen+Shot+2012-03-16+at+3.20.56+PM.png" width="640" /></a></div>
<code><br /></code>Finally you'll reboot.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsLs7Vp2RlPzxBuUS9_Uh7s74f2W7HqBXmt4ZhZjyYvV35ilUfHVzmIsYLeTC7LNWaris4Rn7tkqAbbX9ISJkz_z5yPmSKaOIR7FcuIQz6ksmcJmJg9_UQ3kksy7afAU5ctcIUeKPUbkk/s1600/Screen+Shot+2012-03-16+at+3.24.19+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsLs7Vp2RlPzxBuUS9_Uh7s74f2W7HqBXmt4ZhZjyYvV35ilUfHVzmIsYLeTC7LNWaris4Rn7tkqAbbX9ISJkz_z5yPmSKaOIR7FcuIQz6ksmcJmJg9_UQ3kksy7afAU5ctcIUeKPUbkk/s1600/Screen+Shot+2012-03-16+at+3.24.19+PM.png" /></a></div>
<br />Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com0tag:blogger.com,1999:blog-5361461834637096019.post-9702846206742878642012-02-17T13:40:00.000-08:002012-02-17T14:23:26.085-08:00Get Blogger Posts with NodeJSToday I wrote a blogger scraper for my new NodeJS website: http://mikekunze.info<br />
<br />
This script connects to my blog's RSS JSON feed and sends its data to a mongolab hosted mongoDB instance.<br />
<br />
Pretty nifty. Now I have content for my website, and still can use blogger's API to create that content.<br />
<br />
One of the bonuses with this script is how it will not duplicate entries. This feature checks title names. If we want to be more dynamic we would create hashes on the entry content... maybe some day.
<br />
<br />
<script src="https://gist.github.com/1855612.js?file=getBloggerPosts.coffee"></script>Anonymoushttp://www.blogger.com/profile/02210517104871056369noreply@blogger.com3