Developing a Suite of Mobile Applications for Collaborative Language Documentation Mat Bettinson School of Languages & Linguistics, University of Melbourne Steven Bird Computing and Information Systems, University of Melbourne International Computer Science Institute, University of California Berkeley Context It’s all about collaboration   Documentation calls for a substantial collection of transcribed audio Carrying out this program at scale depends on effective collaboration with speech communities The vision For the monolithic to the prolific “Our vision is for a suite of applications supporting a variety of workflows, and contributing to a common store of language documentation, designed to support the kinds of local and long distance collaborations that arise when working with endangered languages” The vision Shared components “We envisage diverse workflows supported by multiple applications, built and rebuilt from shared components as local requirements evolve” Rolling up the sleeves A year of engineering Aikuma-NG Transcription for normal people*     Can we use web tech to build a decent standalone tool? A Chrome App built with Angular 1.x, ES5, Angular Material, and the bower + grunt build chain We adopted Google’s Material design as a UI, a touch-friendly UI known to literally billions of people Building an offline app was the challenge at the time, Aikuma-NG is 100% offline with local storage * Not Linguists Aikuma-NG Findings   Can we build a real tool? You better believe it! Aikuma-NG broke 700-user mark this morning Component implementation model was unconvincing, inter-comp. communication model is poor (angular 1.x): <ng-player …></ng-player> <ng-language-selector …></ng-player> <ng-tag-selector …></ng-tag-selector> * Not Linguists Aikuma-NG Findings   Can we build a real tool? You better believe it! Aikuma-NG broke 700-user mark this morning Component implementation model was unconvincing, inter-comp. communication model is poor (angular 1.x): <ng-player source="{{sCtrl.audioSourceUrl}}" peaks="sCtrl.audioSourcePeaks" wavecolor="#4FC3F7" progress-color="#33627c" height="120" wavesurfer="sCtrl.wavesurfer"/> <ng-language-selector flex lang-id-name-map="sCtrl.langIdNameMap" lang-idlist="sCtrl.srcLangIds" on-change="sCtrl.saveLangs(langIds)"></ng-language-selector> <ng-person-selector flex role="speakerIds" user-obj="sCtrl.userObj" sessionobj="sCtrl.sessionObj"/> <ng-tag-selector user-obj="sCtrl.userObj" session-obj="sCtrl.sessionObj"/> Template 1way binding * Not Linguists Passing in dependencies in a template Generic Angular change detection events Passing in custom classes of a common data model… E.g. objects with methods to persist to storage Aikuma-NG Engineering in the field Fieldwork tip: Laptop sock dryer! Aikuma-Link A tool for researchers   Linguist returns from the field… … discovers they have incomplete data Oh noes! Speakers are thousands of $unit away, what do we do? Link based onboarding, the crowdsourcing promised land Aikuma-Link First stab at mobile  Crafting a job makes a URL… … which launches a Progressive Web App on the participant’s phone Aikuma-Link Aikuma-Link Findings     Migration to Angular 1.4’s simplified ‘component’ schema, adopted JavaScript ES5, continued with Angular Material Aikuma-Link components take Firebase in component attributes and each component binds the UI to the realtime database In-line with Aikuma-NG, web technologies prove excellent for prototyping mobile UIs The Firebase model is excellent for rapid prototyping but completely fails to extend to offline use cases Adopting Ionic We don’t have to build it ourselves?     Ionic is an open source hybrid mobile framework Features a build chain that uses Apache Cordova to bundle web apps as native apps you can slap on the app stores Ionic 2 uses Angular 2, typescript (typed subset of JavaScript) and Ionic’s own UI which the native UI of iOS and Android (MD) Ionic’s mission is the same as ours: to broaden the base of app developers Zahwa Extending collaboration to the design    We adopted a collaborative app conception and design process – this is vital if you are not the primary user Adopting Ionic freed us to consider problems which had yet to be solved. For example: the issue of how to make a social collaborative app meaningfully useful while the user was offline Zahwa    In ES6/typescript + Angular 2, every UI component, from a button to a whole page, is a class So too are providers but they have no UI and are injected as singleton services The UI class schema differs in entry and exit methods: A list of in-line (atomic) components (recipesnippet) Activity is invoked as a once-off full-screen, and returns some data. Is built from atomic components Component model In-line from a template <lang-edit [languages]="recipe.languages" (langsUpdated)="langsChanged($event)"> </lang-edit>    One-way data binding for a component input Angular 2 Event-emitter scheme for data output In-line component breaks out to modal for comprehensive interaction Component model Programmatic launch of a modal   On mobiles, most activities should be a full-screen modal Activities finish, the UI disappears and the component returns a bundle of data openSnapper(slides: Slide[] = [], displaySlide: number = 0): void { let modal = this.modalCtrl.create(Snapper2Page, { slides: slides, displayImage: displaySlide }) modal.onDidDismiss((data) => { if (data) { this._handleSnapperData(data) } }) modal.present() } Aside from code We need to recruit or train design-savvy people   Web tech makes it easier to build software and prototype UI … but it still needs a certain sort of person to build effective user-interfaces Next steps Where to from here?     We will establish a repository of reusable components We hope to build momentum behind this stack and collaborate on a set of well-crafted components for language-related apps We need a common data API to enable exchange of data between our interconnected apps The purpose of the API is to facilitate a diverse range of language documentation tools which generate, manipulate and mobilize linguistic data Language Data API This is what we will build We are agnostic about how this occurs This is an implementation example using Firebase Fancy a go? A minimal Ionic 2 project      Shell app with some of the needed basics, lashed up for CoEDL TAP workshop Minimal Ionic 2, Firebase federated authorization, real-time database and Google cloud-storage, app localization etc A few basic components: wavesurfer wrapper, file-input component Minimal data service to wrap Firebase in the Reactive Programming pattern of RXJS/Angular 2 https://github.com/Lingomat/WeWrite Thanks for listening! Mat Bettinson mbettinson@unimelb.edu.au @sinomat Steven Bird stevenbird1@gmail.com @StevenBird Aikuma Project research publications http://www.aikuma.org/research.html Zahwa https://zahwa.aikuma.org Aikuma-NG https://goo.gl/1Bt9pp