You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long. 12 KiB

3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
  1. **NOTOC** **Murphy \~**
  2. ## Details
  3. <table>
  4. <tr>
  5. <td>
  6. Type:
  7. </td>
  8. <td>
  9. Sun T2000
  10. </td>
  11. </tr>
  12. <tr>
  13. <td>
  14. OS:
  15. </td>
  16. <td>
  17. <s>Ubuntu 6.06</s> <s>Solaris 10</s> <s>Ubuntu 6.06</s> <s>Ubuntu
  18. 7.10</s> <s>Solaris 10</s> <s>OpenBSD 4.4</s> <s>Solaris 10</s> Debian
  19. Lenny
  20. </td>
  21. </tr>
  22. <tr>
  23. <td>
  24. CPU:
  25. </td>
  26. <td>
  27. 8-core (32 threads) 1Ghz UltraSPARC T1 (Niagara)
  28. </td>
  29. </tr>
  30. <tr>
  31. <td>
  32. RAM:
  33. </td>
  34. <td>
  35. 16GB
  36. </td>
  37. </tr>
  38. <tr>
  39. <td>
  40. Disks:
  41. </td>
  42. <td>
  43. 2x 73Gb 10,000 RPM SAS Disks
  44. </td>
  45. </tr>
  46. <tr>
  47. <td>
  48. Drives:
  49. </td>
  50. <td>
  51. Internal DVD drive
  52. </td>
  53. </tr>
  54. <tr>
  55. <td>
  56. Network:
  57. </td>
  58. <td>
  59. 4x Gigabit Ethernet BroadCom
  60. </td>
  61. </tr>
  62. <tr>
  63. <td>
  64. Extras:
  65. </td>
  66. <td>
  67. Onboard Advanced Lights-Out Management (ALOM) card.
  68. </td>
  69. </tr>
  70. <tr>
  71. <td>
  72. The Name:
  73. </td>
  74. <td>
  75. Named in honour of RedBrick founder David Murphy (drjolt).
  76. </td>
  77. </tr>
  78. </table>
  79. Description
  80. -----------
  81. [thumb\|rightMurphy](/Image:Murphy.jpg "wikilink") was donated to
  82. RedBrick in April 2006 by a member of the society. It is named after
  83. David Murphy (drjolt) who was the founding system administrator of the
  84. society, and who died tragically in January 2006.
  85. The machine is an 8-core UltraSPARC monster. It will gladly chew through
  86. any normal multithreaded application you throw at it (That is not a
  87. challenge). It sucks horrendously at floating-point operations, though.
  88. ## Services
  89. * Primary Web Server
  90. * Available for login
  91. * Planned: Tomcat server
  92. * Solaris <s>keeps the admins on their toes</s> no longer haunts these
  93. lands
  94. ## Brief (hah) history of it\'s OSes
  95. When Sun first released the (somewhat revolutionary) Niagara processor,
  96. Canonical (the people who co-ordinate Ubuntu) made a big deal about
  97. Ubuntu 6.06 fully supporting the architecture. Kernel support for sun4v
  98. (the Niagara architecture) was **very** new, but they said it was
  99. supported.
  100. When we first received murphy, it was nicely set up with Ubuntu 6.06
  101. (thanks to colmmacc). This was great - it had an entire repository of
  102. apt packages, was easy to manage, things made sense, and it was a
  103. long-term support release, so we wouldn\'t have to worry about upgrading
  104. to the next version until 2011. It was also very similar to our other
  105. Debian (and later Ubuntu) machines. Everything seemed to work quite
  106. nicely for the most part. The system went through a few months in a
  107. testing state, then it was promoted to be our login server, due to it\'s
  108. sheer scalability. Everything seemed to be wonderful.
  109. It quickly turned out, however, that users weren\'t too fond of it. The
  110. emphasis of the system\'s design was on multi-threading and data
  111. throughput, but most of our users preferred the single-threaded
  112. performance of carbon. Complaints about speed ensued, even though the
  113. system was never hitting anywhere near max load or memory usage.
  114. Eventually the decision was taken to move primary login to Minerva, our
  115. new storage server with less RAM and inferior scalability but faster
  116. processor clock speeds. It was decided that murphy\'s future would be
  117. best spent as a web server (deathray was crashing once a month because
  118. it just couldn\'t handle the load of WWW anymore anyway).
  119. This was where the problems began to show themselves. MySQL just plain
  120. didn\'t work by default. It turned out to be an issue with libnss,
  121. which, to our knowledge, still hasn\'t been fixed in Ubuntu. We compiled
  122. our own replacement packages, but it was an annoyance.
  123. Apache mostly worked - there were some hiccups but they were mostly to
  124. do with our Apache configuration, not Ubuntu\'s packages. Pubcookie was
  125. a nightmare to set up, but that\'s pubcookie for you. SuPHP worried us -
  126. the packages supplied by Ubuntu were out of date, and had not applied
  127. critical security patches a few months after they\'d been released by
  128. the upstream vendor. We ended up compiling our own patched versions, but
  129. the fact that it had been neglected (whereas up to date versions were
  130. available for Ubuntu 7.10) was concerning.
  131. We moved our web server to Murphy after a few months of testing, and it
  132. worked quite nicely. General response times were a little higher then
  133. they were with deathray (due to the lower CPU clock speed), but the
  134. system never even flinched under heavy load (where deathray would\'ve
  135. locked up).
  136. It was about now that news came that Canonical had dropped SPARC
  137. support. So much for their big noise about sun4v support. Even worse,
  138. they dropped support just months before they released the next LTS
  139. release. It was also about now that Redbrick got rooted, and we had to
  140. reinstall all of our machines from scratch.
  141. Our first course of action was to update the system firmware (as
  142. directed in the Ubuntu install instructions), and put dapper back on
  143. there. 2011 was a long way away, we could worry about rearchitecture
  144. later on. However, it turned out that the newest version of the firmware
  145. wouldn\'t even boot Linux, which was very worrying. Surely canonical
  146. would\'ve at least thought to release a kernel update that ran on the
  147. updated hypervisor (Sun4v runs everything inside a hypervisor).
  148. We faced a descision as to what to do with Murphy. There were a few
  149. alternatives, none of which were too appealing.
  150. * Install Ubuntu 6.06 again, knowing that Canonical seemed to be just
  151. losing interest in it (leaving out security patches, not bothering
  152. to patch libnss, leaving the kernel too old to boot on modern
  153. firmware). This would involve downgrading the firmware again.
  154. * Install Ubuntu 8.04 from ports. This is an unofficial Ubuntu
  155. release - essentially, a bunch of build scripts are set up to
  156. compile everything for SPARC, and it\'s left to do it\'s own thing.
  157. If things break, nobody really cares too much. While in theory
  158. everything here is newer and should work better, there\'s never a
  159. guarantee that anything will work.
  160. * Install Solaris 10. Solaris has near perfect support for Sun4v,
  161. because Sun has a very clear vested interest in it. However, Solaris
  162. is completely devoid of package management, making administration a
  163. nightmare. Also, it\'s full of proprietary crap that\'s incompatible
  164. with everything else we\'ve spent years putting together. Finally,
  165. none of the current elected root holders had ever used Solaris.
  166. * Install Debian. There was evidence that it ran on Sun4v, but three
  167. hours of googling revealed 3 (yes, **three**) people in the world
  168. that seemed to have even tried it.
  169. * Install Gentoo. Seemed to have more users then Debian did, and the
  170. community around it seemed more interested in keeping it up to date
  171. then Canonical did with Ubuntu. However, a number of root holders
  172. seemed to object to \"gen-feckin-too\" (an actual quote), and more
  173. seriously, it doesn\'t have the enforced package compatibility that
  174. Ubuntu/Debian does. In other words, an update one day could cause
  175. our apache configuration to suddenly break for unknown reasons.
  176. * Install a BSD variant. This had a smallish community around it, but
  177. again administering it is more of a pain then the average Debian
  178. system, there was little in the way of root holder previous
  179. experience (although more then there was for Solaris), and there
  180. didn\'t seem to be any amazing success stories of it being used on
  181. servers in the wild.
  182. We decided to stick with the devil we knew at first. We downgraded the
  183. firmware and put Ubuntu (6.06) back on it. We quite quickly regretted
  184. this - running into bizarre problems ranging from the serial console
  185. randomly not responding to random segmentation faults. The installer
  186. also didn\'t work at all; werdz ended up installing it manually with apt
  187. from a rescue console.
  188. This was fairly unacceptable - especially the issue with the serial
  189. console. The ALOM still had the ability to shut off system power and get
  190. to OpenBoot, etc, but we couldn\'t actually access the operating system.
  191. So we jumped ship to Solaris.
  192. As expected, it installed fairly painlessly (the installer wasn\'t
  193. amused at the sight of Linux partitions on the disks, but aside from
  194. that it worked perfectly). Once it was booted, everything worked nicely.
  195. It had far better support for the hardware than Linux did - the amount
  196. of detail given by prtdiag in the global zone showed that off. It could
  197. even update the firmware itself!
  198. Another nice feature was zones/containers. We decided to split the
  199. system into a web host zone (with user logins), another for databases,
  200. and another to run a secondary LDAP in.
  201. Unfortunately, getting Solaris to work with the rest of our systems was
  202. painful. Cian first tried getting PAM and PADL to authenticate off our
  203. LDAP, but couldn\'t get them to compile. After a while spent fecking
  204. around with the Sun client and a few other LDAP servers (trying them
  205. out), we eventually (somehow) got PADL working. Hurdle number 1
  206. conquered.
  207. The web stack proved to be equally as nightmarish. It took a while to
  208. even get apache to compile (turned out to be a path issue, I felt
  209. stupider then usual that day). PHP was a nightmare, because of the
  210. number of dependencies that had to be either installed from Sunfreeware
  211. or compiled. What we\'d have done for apt-get install php5. That
  212. eventually compiled.
  213. SuPHP compiled, but it doesn\'t work. And now, for no absolutely reason
  214. at all, LDAP authentication has broken again on it. Grumble grumble.
  215. The system is also a nightmare to keep up to date. It\'s a pain that we
  216. have to do it alright, but what really worries us is that it will
  217. stagnate when the next group of rootholders are elected.
  218. : As such, it\'s entirely possible that I\'m going to end up kicking
  219. the shit out of Murphy one day, and regardless of my reasoning,
  220. it\'d deserve it. [Coconut](/User:Coconut "wikilink") 01:02, 16 May
  221. 2009 (UTC)
  222. So we\'re seriously considering the other alternatives again. Ubuntu
  223. 8.04 seems worth a try, but the main problems with that are likely to
  224. come down the line, when package builds start failing and nobody does
  225. anything about it. The only other reasonable option seems to be Gentoo,
  226. which makes some root holders break out in hives at the very mention of
  227. it\'s name. \--[Werdz](/User:Werdz "wikilink") 23:54, 1 Sep 2008 (IST)
  228. Eventually, after lots of cursing, and werdz getting a good proportion
  229. of the way into porting [Nexenta]( to sparc,
  230. we decided to move to OpenBSD. It\'s a system that myself and werdz have
  231. a reasonable amount of experience with, it has a reasonable packaging
  232. system (but as phaxx said, apt was spoiling us anyway), just work (TM)
  233. on Sun_4v, seems to be well looked after, and is just generally
  234. amazing. It\'s lack of PAM/NSS support is a bit of a pain, but YPldap
  235. appears to be doing the job, without any major hicups. We\'re still
  236. compiling our own apache, but that\'s a minor thing compared to the
  237. million and one things we\'d have to look after on solaris.
  238. \--[lil_cain](/User:lil_cain "wikilink") 14:16, 24 Nov 2008 (IST)
  239. OpenBSD worked quite nicely. Until we tried to actually run stuff on it
  240. in production. Within minutes of the Apache changeover, it was kernel
  241. panicing like mad :(. Looks like sun4v support isn\'t quite there yet.
  242. So our solution was to try Solaris 10 again, and we think we\'ve gotten
  243. it just right this time. We built our own packager from scratch (800
  244. lines of delicious Perl) to keep things like Apache, suPHP, PHP, and all
  245. of its lovely dependencies up to date and running properly
  246. automatically. We\'ve just gotten apache itself configured and running,
  247. and hopefully in the not too distant future we\'ll have it running in
  248. production. Also, we beat the native LDAP client into working, so no
  249. dirty OpenLDAP haxes. \--[Werdz](/User:Werdz "wikilink") 00:48, 5 April
  250. 2009 (UTC)
  251. Well, Solaris worked happily (ish) for a while. It held the web server
  252. for a while, we experimented with various ways to keep it up to date
  253. easily, etc, but in the end just couldn\'t get it to a stage where you
  254. could update it with less then a day\'s preparation and the patience of
  255. a saint. So on the 20th of February 2010 we bit the bullet and put Linux
  256. back on it. And it\'s working. Perfectly. Thank fsck.
  257. \--[Werdz](/User:Werdz "wikilink") 21:00, 21 February 2010 (UTC)
  258. [Category:Hardware](/Category:Hardware "wikilink")