O’Reilly Media. Help­ing an Or­ga­ni­za­tion Rein­vent It­self Through the Years.

For al­most 40 years, O’Reilly has pub­lished world-shap­ing ideas about dig­i­tal tech­nol­ogy. Tra­di­tion­ally seen as a book pub­lisher, the com­pany has rein­vented it­self as a fully fledged me­dia or­ga­ni­za­tion, dis­trib­ut­ing their con­tent as books, ar­ti­cles, train­ing courses, con­fer­ences, and through an on­line learn­ing plat­form. Dur­ing our longest run­ning part­ner­ship, we have helped O’Reilly with sev­eral high-stake pro­jects. Here, we will dive into four of the ma­jor pro­jects we have taken part in.

Publishing System 2014

Writ­ing a book is a com­pli­cated process. First, an au­thor strug­gles for months or years to write it. Then, a pro­duc­tion ed­i­tor man­u­ally puts the con­tent into an ap­pro­pri­ate de­sign for both print and e-book for­mats. It is a slow and te­dious wa­ter­fall process that leaves much to be de­sired: Au­thors can­not see what their book looks like un­til the very end. Word files are emailed back and forth. Pro­duc­tion ed­i­tors have to re­peat a sisyphean process when­ever a sin­gle comma changes.

Every edit made in At­las is au­to­mat­i­cally saved, which makes it easy for au­thors to track changes made by ex­ter­nal ed­i­tors.

In 2014, we set out to rein­vent con­tent pro­duc­tion at O’Reilly. The re­sult was Atlas, a web-based ap­pli­ca­tion that al­lows au­thors and pro­duc­tion ed­i­tors to work in the same in­ter­face. Every edit made in At­las is au­to­mat­i­cally saved, which makes it easy for au­thors to track changes made by ex­ter­nal ed­i­tors. By the push of a but­ton, a book can be ex­ported into sev­eral pro­duc­tion-ready for­mats for print, e-read­ers, and the web. Af­ter five years in pro­duc­tion, At­las has been used to pro­duce more than 1500 books, sav­ing thou­sands of hours of pro­duc­tion time.

00:00
By the push of a button, a book can be exported into several production-ready formats for print, e-readers, and the web.

The At­las plat­form con­sists of sev­eral pieces of soft­ware. The most promi­nent is the web ap­pli­ca­tion, built with Ruby on Rails and Back­bone.js, which ex­poses the user in­ter­face. In this ap­pli­ca­tion, book con­tent is au­thored in HTML or Mark­down us­ing ei­ther the WYSI­WYG or plain text ed­i­tor. Like many web frame­works, At­las sep­a­rates con­tent and pre­sen­ta­tion: A cus­tom them­ing frame­work al­lows styling of books in CSS, which means that the same con­tent can be used to gen­er­ate many vari­a­tions of a book in dif­fer­ent lengths and de­signs, and changes in con­tent are au­to­mat­i­cally re­flected in the lay­out. The ser­vice uses Git repos­i­to­ries to ver­sion-con­trol book con­tent via a sep­a­rate server run­ning Git­lab, the open source Git-host­ing plat­form. A third sys­tem, a set of back­ground processes, are re­spon­si­ble for trans­form­ing book con­tent, ap­ply­ing the theme en­gine, and ex­port­ing it into PDF, ePub, MOBI and HTML for­mats.

The same con­tent can be used to gen­er­ate many vari­a­tions of a book in dif­fer­ent lengths and de­signs, and changes in con­tent are au­to­mat­i­cally re­flected in the lay­out.

These processes in­tel­li­gently clone and cache the Git repos­i­to­ries, run a cus­tom set of trans­for­ma­tions writ­ten in JavaScript and XSLT, and up­load the fi­nal for­mats to S3 stor­age. This setup al­lows non-tech­ni­cal au­thors to write in an in­ter­face close to that of Word or Google Docs with­out need­ing to know any­thing about Git, whereas more tech­ni­cal au­thors and ed­i­tors can ap­proach their book writ­ing sim­i­lar to what they know from soft­ware de­vel­op­ment, us­ing Git to write in a lo­cal text ed­i­tor and push the up­dates to At­las via the com­mand line.

New Website and Design System 2015

Af­ter a suc­cess­ful launch of At­las, O’Reilly wanted to fo­cus on a ma­jor re­vamp of or­eilly.com. Their web­site was pri­mar­ily an in­de­pen­dent on­line book­store — a big suc­cess story in the age of Ama­zon, but it did not re­flect the full range of prod­ucts that the com­pany of­fered. Worse, a decade of fea­ture im­prove­ments and quick fixes had left it in a state that —aes­thet­i­cally and func­tion­ally— did not sup­port the brand. The com­pany who cre­ated the world’s first com­mer­cial web­site needed a makeover.

In col­lab­o­ra­tion with Josh Clark and Dan Mall, we cre­ated a new vi­sual lan­guage, in­cor­po­rated this into a de­sign sys­tem, and built a new in­fra­struc­ture for or­eilly.com. While the in­tent stayed the same —to spread the knowl­edge of in­no­va­tors— the web­site took a new ap­proach by show­cas­ing ar­ti­cles, videos, long-form re­ports, and in­ter­ac­tive con­tent on cu­rated topic pages that could be fol­lowed by users.

With a sim­ple im­port state­ment, teams were now able to use the new col­ors, ty­pog­ra­phy, nav­i­ga­tion and lay­out struc­tures in their ap­pli­ca­tions.

A re­design of a web­site from 1997 with mil­lions of weekly users is not a sim­ple mat­ter. The lack of a cen­tral de­sign repos­i­tory had re­sulted in code du­pli­ca­tion across a range of legacy sys­tems, each re­spon­si­ble for a sub­set of the web­site. These sys­tems had their own meta­data han­dling and no ex­ter­nal API’s, which made it near im­pos­si­ble to use their data on other parts of the plat­form. This also caused ma­jor in­con­sis­ten­cies be­tween dif­fer­ent sec­tions, both in the vi­sual de­sign and in­ter­ac­tion pat­terns.

The first ma­jor mile­stone was to trans­late the new vi­sual lan­guage to a set of reusable com­po­nents to be dis­trib­uted in a cen­tral­ized pack­age. We opted for a low-tech setup, us­ing Han­dle­bars.js tem­plates that could be used by both a server (Python, Go, Ruby, Node, etc) and in a web client via Back­bone.js. With a sim­ple im­port state­ment, teams were now able to use the new col­ors, ty­pog­ra­phy, nav­i­ga­tion and lay­out struc­tures in their ap­pli­ca­tions. We cre­ated a data-ex­trac­tion task and an API gate­way to col­lect, shape and ex­pose data from the legacy sys­tems in a com­mon end­point. On top of this, we cre­ated a new back­end CMS to cu­rate this data for use on the new web­site. The web­site it­self is served by a new web server writ­ten in Go to sup­port mas­sive traf­fic loads with im­pres­sively low la­tency while keep­ing a small server foot­print. We also cre­ated a cus­tom an­a­lyt­ics dash­board for ed­i­tors to eval­u­ate the suc­cess of their con­tent and ex­ec­u­tives to eval­u­ate the over­all suc­cess of the plat­form.

00:00
The system translated the new visual language to a set of reusable components to be distributed in a centralized package.

The re­sult is a fast and re­spon­sive web­site that re­flects the val­ues of the brand and sup­ports the needs of the busi­ness with­out hav­ing to wait for sub­stan­tial changes to back­end legacy sys­tems.

New Learning Products 2016

In early 2016, we be­gan a re­search pro­ject fo­cused on new ways of reach­ing O’Reil­ly’s au­di­ence through the new web­site. Most tu­to­ri­als on or­eilly.com used a sin­gle medium —of­ten text or video— to ex­plain tech­ni­cal con­cepts. Al­though this con­tent is pop­u­lar, these for­mats can be prob­lem­atic be­cause the learner is re­quired to go through a com­pli­cated in­stal­la­tion process on their own com­puter be­fore a line of code can be writ­ten. O’Reilly had al­ready solved this prob­lem at their con­fer­ences where at­ten­dees re­ceive ready-to-go vir­tual en­vi­ron­ments to start cod­ing im­me­di­ately. The goal of the pro­ject was to em­u­late this ex­pe­ri­ence on the web.

Ori­ole is a unique new medium that blends code, text, and video into a nar­rated learn­ing ex­pe­ri­ence, ex­e­cut­ing code in a clus­ter of Jupyter note­books di­rectly from a browser.

The re­sult was Oriole, a unique new medium that blends code, text, and video into a nar­rated learn­ing ex­pe­ri­ence with ex­e­cutable con­tent. Led by some of the most bril­liant minds in tech­nol­ogy, each les­son is an eas­ily di­gestible and en­gag­ing thought-by-thought tour of the in­struc­tor’s ap­proach to the prob­lem in both nar­ra­tive and ex­e­cutable code. Users can write and ex­e­cute code di­rectly in the in­ter­face, and it re­quires noth­ing more than a browser and an in­ter­net con­nec­tion.

00:00

We cre­ated the Ori­ole in­ter­face with Re­act.js, the Kaltura li­brary for con­trol­ling videos in real-time via JavaScript, and a cus­tom JavaScript li­brary for ex­e­cut­ing code in a clus­ter of Jupyter note­books di­rectly from a browser. This li­brary sends the code over a socket, ex­e­cutes the code on a Jupyter ker­nel, and sends any out­put or er­ror mes­sages back through the socket con­nec­tion. This al­lows de­vel­op­ers to cre­ate fully cus­tomized ex­pe­ri­ences on top of the Jupyter frame­work.

Safari Design System 2018

When the on­line learn­ing plat­form Sa­fari be­came a wholly owned sub­sidiary of O’Reilly Me­dia in 2014, Tim O’Reilly wrote that “there are sub­stan­tial op­por­tu­ni­ties for both or­ga­ni­za­tions work­ing much more closely to­gether”. Af­ter sev­eral years as a sep­a­rate en­tity, Sa­fari was em­braced as an ex­ten­sion of the O’Reilly brand in 2017, which had to be re­flected in the look and feel. At the same time, Sa­fari was ea­ger to tran­si­tion from a wa­ter­fall process with lit­tle in­ter­ac­tion be­tween de­sign­ers and en­gi­neers to a more col­lab­o­ra­tive process cen­tered around a solid de­sign sys­tem.

Over four months, we worked with an in­ter­nal team at O’Reilly to cre­ate a new de­sign sys­tem and a doc­u­men­ta­tion site with re­sources and best prac­tices for us­ing the com­po­nents. We also helped es­tab­lish a team of de­sign­ers and en­gi­neers re­spon­si­ble for the con­tin­ued de­vel­op­ment of the de­sign sys­tem, as well as a work­flow where de­sign­ers are di­rectly in­volved in the de­vel­op­ment ef­forts of the com­po­nents.

00:00
To­gether with O’Reilly's in­ter­nal team, we cre­ated a new de­sign sys­tem and ac­com­pa­ny­ing doc­u­men­ta­tion site.

The core of the de­sign sys­tem is a set of com­pos­able Re­act.js com­po­nents that also bun­dle into a stand-alone CSS file for use in sim­pler client ap­pli­ca­tions. The doc­u­men­ta­tion site is gen­er­ated di­rectly from the Re­act.js source where a com­po­nen­t’s Prop­Types and in­line com­ments are parsed into a read­able struc­ture and used to gen­er­ate a sta­tic site for the com­po­nent. With the de­sign sys­tem also comes a sep­a­rate pro­ject gen­er­a­tor script sim­i­lar to cre­ate-re­act-app, mak­ing it ef­fort­less to spin up a new ap­pli­ca­tion set up with the de­sign sys­tem. Pro­jects gen­er­ated with the tool au­to­mat­i­cally have a stream­lined de­vel­op­ment en­vi­ron­ment, server-side com­po­nent ren­der­ing, and a mock API for data load­ing. This not only al­lows de­sign­ers and de­vel­op­ers to quickly pro­to­type new ideas, it also saves sig­nif­i­cant time dur­ing de­ploy­ment since every pro­ject is gen­er­ated with the plumb­ing needed to run on Sa­far­i’s Ku­ber­netes pro­duc­tion setup.