It's easy to complain about bad tools. It's harder to write a great tool yourself. Once you step into the shoes of an internal tools developer, you realize that you're balancing feature requests and constraints from a dozen different teams and an entire universe of complexity behind what should be a simple UI. In this talk, we'll talk about how to gather information about your users' needs, how to balance between conflicting requirements, and ways you can fail to build a tool that your coworkers want to use.

Black-box testing methods such as Jepsen have toppled many guarantees such as strong consistency and snapshot isolation promised by distributed SQL and SQL-like databases. Such implementations are difficult to get right, and often wrought with errors. Accordingly, there has been an increase in the use of such black-box testing tools to ensure consistency guarantees are upheld during development. The increased interest in Jepsen is evidence that traditional software quality approaches such as formal verification and white-box testing are inadequate for these systems due to their complexity. However, though this solution is quite attractive, the question arises as to whether the tools themselves are correct.This talk discusses a generative method for assembling nontrivial, non-serializable histories to test these black-box serializability checkers rooted in theoretical explorations of consistency anomalies.

Born in a lab, matured in the enterprise, and now fully baked as an open-source, cross-platform, professionally-supported language: the F# journey has always been about reconciling the apparently irreconcilable. Javascript and .NET, Linux and Windows, Functional and Objects, Company and Openness, Android and iOS, Programming and Data, Server and Client - F# finds a way. Come along and take a journey with me through the modern programming landscape and the F# approach to language design, interoperability, tooling and community.

Every-time there is a new piece of big data technology we often see many different specific implementations of the concepts, which often eventually consolidate down to a few viable options, and then frequently end up getting rolled into part of another larger project. This talk will examine this trend in big data ecosystem, look at the exceptions to the "rule", and look at how better interchange formats like Apache Arrow have the potential to change this going forward. In addition to general vague happy feelings (or sad depending on your ideas about how software should be made), this talk will look at some specific examples with deep learning, so if anyone is looking for a little bit of pixie dust to sprinkle on a failing business plan to take to silicon valley to raise a series A, you'll get something out this as well.

Over the years there have been many good ideas in computer science and many stupid ideas. What can we learn from history? Which of the old and forgotten ideas should be brought back to life and re-evaluated? What can we learn from the stupid ideas of the past?

Like all imperative languages Java is, with some minor but notable exceptions, an eagerly evaluated programming language. Nevertheless the introduction of lambdas in Java 8 also allowed the adoption of some lazy patterns and data structures that are more typically employed in functional languages. Streams represent the most evident example of how also native Java API has taken advantage of laziness, but there is a number of other interesting scenarios where laziness can be an effective solution to quite common problems. In fact laziness is the only possible technique to process potentially infinite amount of data, or more in general to delay the expensive evaluation of an expression only when and if it is necessary. But laziness is even more than that: for instance the reader monad delays not only a computation but also the need of external dependencies thus lowering the abuse of dependency injection, while a trampoline uses laziness to delay and then linearize recursive calls preventing the overflow of the stack. The purpose of this talk is illustrating why and how implementing laziness in Java with practical examples delivered with both slides and live coding sessions.

Want to know what is possible in the field of high performance consensus algorithms for distributed systems? If the answer is yes then come to this talk.

Once upon a time, we used software that ran on our own computers, that worked offline, and that stored its data in files on the local disk. Then we decided to put it all in the cloud. We gained some great features: real-time collaboration, like in Google Docs, for example. But we also lost control of our own data, and became dependent on far-away servers to allow us to access the data that we created. Automerge is part of an effort to get the best of both worlds. It is a JavaScript library for building real-time collaborative applications. However, apps built with Automerge also work offline, storing data locally, and synchronise their data with collaborators whenever a network is available. And although you can use it with servers, you don’t have to: synchronisation also works peer-to-peer, or via any network you choose. In this talk we will explore how Automerge deals with different users independently modifying shared data in a collaborative application (hint: by merging the changes... automatically!), how it achieves consistency in highly distributed settings, and where it is heading in the future.

Terraform enables you to describe almost anything with an API as code. From infrastructure to high level services, Terraform can help construct everything required to run your applications. In this talk, we’ll explore using Terraform to manage both traditional and non-traditional resources and dive into how to write best practices Terraform for reusability within teams.

On-line transaction processing (OLTP) database management systems (DBMSs) are a critical part of the operation of many large enterprises. These systems often serve time-varying workloads due to daily, weekly or seasonal fluctuations in demand, or because of rapid growth in demand due to a company’s business success. In addition, many OLTP workloads are heavily skewed to “hot” tuples or ranges of tuples. For example, the majority of NYSE volume involves only 40 stocks. To deal with such fluctuations, an OLTP DBMS needs to be elastic; that is, it must be able to expand and contract resources in response to load fluctuations and dynamically balance load as hot tuples vary over time. In this talk, I will present a system we built at MIT called P-Store, which uses predictive modeling to reconfigure the system in advance of predicted load changes. I will show that when running a real database workload, P-Store achieves comparable performance to a traditional static OLTP DBMS while using 50% fewer computing resources.

If Java 8 was all about how we code, Java 9 is all about how we will build. Modularization will have the biggest impact of any change that happened in Java since its inception. In this presentation we will learn about the need for modularization, how it impacts development, the rules to follow when creating modules, and the effect it has on legacy code. We will explore creating module, using modules, readability, exports, automatic modules, and unnamed modules.

Operators of large distributed systems often try to optimize for scarce computational resources — memory, network bandwidth, and CPU cycles. However, research shows that if we want to maximize the health of our systems, there’s another scarce resource we should be prioritizing instead: the attention of our users.

In this talk, I’ll show how we can use machine learning to improve how we present data and information and tell a story to our users, optimizing for their attention and holistic understanding, instead of overwhelming them with a barrage of low-level metrics. More intriguingly, we’ll see how this approach has produced massive improvements in several real-world systems at large Fortune 100 institutions. By the end of the talk, we’ll hopefully have convinced you that happier and more informed operators isn’t merely a good outcome, but rather is the most ethical thing to do if we don’t want to lead people astray.

Transactions and Concurrency Control are very of paramount importance when it comes to enterprise systems data integrity. However, this topic is very tough since you have to understand the inner workings of the database system, its concurrency control design choices (e.g. 2PL, MVCC), transaction isolation levels and locking schemes.

In this presentation, I’m going to explain what data anomalies can happen depending on the transaction isolation level, with references to Oracle, SQL Server, PostgreSQL, and MySQL.

I will also demonstrate that database transactions are not enough, especially for multi-request web flows. For this reason, I’m going to present multiple application-level transaction patterns based on both optimistic and pessimistic locking mechanisms.

Last, I’m going to talk about Concurrency Control strategies used in the Hibernate second-level caching mechanism, which can boost performance without compromising strong consistency.

In 2018 we will see a big change in the IoT landscape: based on LTE-m devices can connect to the internet over long distances and will be running off batteries for years. Learn about the principles and protocols involved and how to leverage JavaScript down to the hardware to build your own solution.

The software industry has spent the last 10 years creating tools and frameworks that reduce the friction in the delivery process, like continuous integration tools, test driven development, automated deployments, etc. There should not any reason why projects that have hardware components cannot use the same tools. However, in practice, this process can be really costly and risky and it requires a mindset that hardware and embedded system developers are not used to.

In this talk, I explain how to apply continuous delivery practice in projects that contain hardware and software components and embedded systems and how to solve the most common challenges.

Main topics include:

• Strategy for creating continuous integration pipelines
• Tips on how to reduce the development feedback cycle
• How to automated deployments and test on physical boards
• What development practices work in this environment

Have you ever wondered how to actually achieve elasticity on the cloud, outside of your cushy local dev environment?

Akka is the most mature choice to implement the traits of the Reactive Manifesto, thanks to the Actor model. Resilience is enabled by asynchronous message passing and Actor supervision. Location transparency, message routing and the “share nothing” approach enable seamless scalability, but once our app is deployed we need to rely on some external infrastructure to automatically scale up or down our services. At this point, Akka alone is not sufficient anymore. What else do we need?

We found Docker & Kubernetes to be a perfect match for clustered Akka applications running in the cloud. Thanks to the Kubernetes API, one can write infrastructure logic that complements the business logic. This leads to a general approach of infrastructure-as-code tailored to each different business case, which in turn lets companies achieve maximum efficiency and resource usage.

During this talk we will cover features from both Akka and Kubernetes, plus example code in Scala: familiarity with these technologies is recommended. Our example will rely on Google Cloud infrastructure.

The rise of microservices has allowed ML systems to grow in complexity but has also introduced new challenges when things inevitably go wrong. Most companies provide isolated development environments for engineers to work within. While a necessity once a team reaches even a small size, this same organizational choice introduces potentially frustrating dependencies when those individual environments inevitably drift. Kinnary Jangla explains how Pinterest dockerized the services powering its home feed to accelerate development and decrease operational complexity and outlines the benefits Pinterest gained from this change that may be applicable to other microservice-based ML systems. This project was initially motivated by challenges arising from the difficulty of testing individual changes in a reproducible way. Without standardized environments, predeployment testing often yielded nonrepresentative results, causing downtime and confusion for those responsible for keeping the service up.

The Docker solution that was eventually deployed prepackages all dependencies found in each microservice, allowing developers to quickly set up large portions of the home feed stack and always test on the current team-wide configs. This architecture has enabled the team to debug latency issues, expand its testing suite to include connecting to simulated databases, and more quickly do development on our thrift APIs.

Kinnary shares tips and tricks for dockerizing a large-scale legacy production service and discusses how an architectural change like this can change how an ML team works.