An Technical E-book for a Software Company
AN OUTSIDE-IN PERSPECTIVE OF API FRAMEWORK
The Outside-In Perspective
The outside-in perspective is a design and development philosophy that focuses on a customer’s outside view and then flows towards a company’s inside view to define products and application experiences. Traditionally, companies designed applications and specifically APIs (Application Programming Interfaces) with an inside-out perspective, where the business would determine the type of data the company had to offer, and then expose that data for application developers to figure out how to use it. In today’s dynamic and disruptive marketplace, companies must implement an outside-in perspective to not only stay relevant with their customers, but to quickly adapt to the continuously changing marketplace.
The prerequisite to inventing amazing digital products is being thoughtful about the needs of customers which rather imply generating feedback loops and flexibility in changing plans when the needs of customers change. The reason why current businesses must accept end users and their developers as clients is because the developers involved are highly valued and sought after. Web Developers need accessible backend systems which they have a full control over in order to carry out their jobs. In effect, developers utilize a set of rules or software interactions that allows two applications to communicate. These programming interfaces are known as APIs (Application Programming Interfaces). APIs that are well designed extract the intricacy of systems behind a spontaneous interface that is serviceable for developers. APIs that uncover systems and are not workable for developers can hinder the progress of a business and also generate products that do not satisfy the various needs of customers. User-experience and digital products need a business model that is configured according to outside-in perspective of APIs and this is essentially what customers and developers want. Most organizations utilize both inside-out and outside-in, however, with the consideration of user experience and better delivery, they are inclining towards the adoption of outside-in perspective for efficiency and fluidity.
In any commercial undertaking, the ideology of outside-in is gradually gaining prominence as the amalgamation of social CRM—community relationship management and design-centric business style has evolved into a paradigm shift. Innovations are effortlessly combined with the patterns of communication between companies and clients. The upsurge of social web applications and domains, social computing, creation of franchise and personalization also gives consumers a clear understanding from different perspectives. The more a consumer understands how the system works, the more the business expands to offer better personalized services.
The development of technology continues to upset the traditional way of doing things for most businesses. Digital business through mobile commerce—which includes the use of Bluetooth, wireless, text messages and cloud computing systems are taking over the global market. Small-scale enterprises are showing the aftermath of collision, with the behavior of new users, on the ecosystem. Consumers want a comfortable life; hence, they desire a multi-purpose device which can ease trading, transactions and delivery of products/services. Payment through cell phones facilitates transactions and removes the necessity of keeping a wallet. With the changes in market trends, it is advisable to know or get acquainted with your customers. When there is a decline in income, it is expected to concentrate on short-term tactics that reduces cost; however, an internal factor can incapacitate your business with time. What establishments should do is to channel their efforts towards external trends, modern technologies, consumer behaviors that effecting a change in the industry. Some simple facts that propelled API outside-in perspective in recent businesses are;
Customers are more curious
Customers research virtually everything. The reality is that we cannot have a full knowledge of everything. Here are some other insights that show these behavioral changes. Over the past two years, we discovered that;
Searches for the term “best” grew over 80% on mobile.
Videos with “review” in the title had more than 50,000 years’ worth of watch time on mobile alone
Watchtime of “does it work?” videos grew by more than 1100%
49% of users visit 2-4 websites before converting
Customers are more demanding
Starting in early 2017, search volume for local searches without ‘near me’ have outgrown comparable searches that included 'near me.
Customers now expect Google to understand ‘near me’ searches whether its typed in a search or not, relying on machine to fill in the blank
63% of people expect brands to provide personalized experiences based on past interactions.
Customers are more impatient
Over the past two years, travel-related searches for “tonight” and “today” have grown 150% on mobile (for example, “flight today,” “hotels tonight”)
Mobile searches for “wait times” grew over 120% over the last 2 years
Today’s customer is more empowered than ever before. And with the growth in connected devices, this empowerment will only deepen. This new behavior is having a significant impact on the world of technology.
This ebook will address several areas of an organization and how an outside-in perspective can help your business meet the needs of your customers, grow your business and digitally transform your organization.
Agile Methodology
Agile software development refers a system of software development processes centered on an iterative method of developing and designing programs. This is in contrast to software development that involves extreme planning and advance scheduling. Agile development implies that tasks are fragmented, appraised and developed as they are concluded, while the clients and involving investors are able to see a minor part of the concluded results.
The main purpose of an agile methodology is to create and design faster software. This method delivers a lightweight structure for assisting teams. It makes them functional and lays emphasis on rapid delivery of product/service. This methodology makes organizations more agile in product/service delivery to customers thereby minimizing the risks associated with the development of software. It fast-tracks development because many developers do modifications which is later amalgamated into the product and released repeatedly as iterations. This process is done daily by one of the biggest social networking sites—Facebook. Intermittent upgrading of software enhances the quality and functionality of product.
Many companies are now embracing agile methodology and continuous integration/continuous delivery (CI/CD) throughout the organization. The usage of Waterfall methods is no longer in vogue as businesses and industries continue to evolve. It is noteworthy that continuous deployment coupled with some processes can allow introduction of new performance. However, it is crucial to integrate completely programmed tests into code management process. An API solution is complicated in a way and it has to be managed by an individual. This intricacy can either by managed by the client or by the API. If the intricacy is managed by the client side, the assignment of the API provider is easy while that of the API consumer is problematic. This is the outcome of an inside-out perspective and it usually results in substandard and procrustean APIs. If the intricacy is managed by the API side, the API provider encounters a Byzantine task while the API customer handles a simple task. This is the outcome of outside-in perspective. This outside-in perspective has a greater possibility of generating APIs that customers appreciate and despite the challenges involved for the API provider, this is the best and recommended strategy.
The modern technologies of big data systems, mobility, and cloud computing allow the effective application of this method without breaching any business model. An existing effective practice is to invent and organize a broad interconnected IT environment in a confined safe state that centers on data communication between client and servers. But it is noteworthy that the arduous task of allowing anonymous personnel and haphazard operations of networks to gain entry into a secure system cannot be overemphasized. As a result, the swift and intermittent introduction of modern “apps” or “services” that enable communication via social networks and collaboration tools, combining voluminous data, and the deployment phases of modern apps can become strenuous processes.
In embracing an ‘outside-in’ system, the important users and devices are relocated outside the current protected IT environment. In the case of the airport, operations will exist side-by-side on a cloud that allows a system with distributed memory and dependent components, stateless, diminution of ‘services’ backed by this shared cloud. The random need to check email, and a few business applications can be managed with real thin clients operating from ‘outside-in’, thus avoiding the exposure of data, network or application in the same way as if these devices and consumers were operating from ‘inside-out’ and demanding that everything should be ‘conveyed inside’ the secure zone. This model can also embrace customers bringing their personal devices, agile business and other means by which the condition for a new business model is stated.
The agile methodology in businesses is used to administer commercial processes in a highly volatile situation. The way businesses use agile processes to react to variations in the organization to enhance productivity and satisfy the sudden needs of consumers is similar to the way developers use agile processes to react to variations in project conditions.
Agile businesses give importance to mobility and quick decision-making, rather than depending on conventional bureaucracy. A few easy steps can kick start the agility of any business;
Creating teams across-the-board and making them resolve particular problems, rather than permitting divisions or business units to relay duties concerning issues among themselves.
Adopting mobile-based and cloud-based technology and permitting staffs to work outside conventional office locations.
Working to truncate project time schedules so that the organization can augment its products or services swiftly as a feedback mechanism to fluctuations in the market.
Essentially, both software development companies and business enterprises need to embrace the Agile methodology in order to keep up with the successful execution of project and also development ecosystem. However, the satisfaction of customers’ need, analysis and dialogue between developers and end users is very important. This process also helps to promote integrity and trust between end users and developing teams. In choosing Agile, one is assured of the best software solutions that give the finest delivery and minimalism.
Analysis
Domain Modeling and Event Storming are both amazing tools used for creating API stories from the business angle. We need to describe an equitable scope for these processes so that we can prevent complicated modeling strategies and failed delivery.
Event storming is essentially beneficial if you have the knowledge that all the potentials of the business are being pushed into a REST API. REST is not a broad solution because it is a single tool that allows technical communication. REST presents itself in a good way to response-response needs while Event storming helps to force out needs for conveying communication to messages and events. APIs that supports the integrity of business comprises of information, behaviors disclosed by REST, and even events. This concept flows back to the fundamentals of object-oriented programming, as most of the business objects comprises properties, events, and methods which is in agreement with REST. An analysis that is inclined towards the business angle can give a “call to action” when an API strategy is incorporated with ‘just build a REST interface around our existing systems’.
The first step to any application or brand design is having a distinct knowledge of what customers and internal investors want. When your organization chooses to unmask a public API to facilitate interaction with consumers or other companies, you have to comprehend the needs and the driving force of the primary business. Effective public APIs have well-defined themes and are usually copyrighted. For instance, Best Buy has API with a definite messaging that describes the aim of each API domain and it is easy for a novice to know the reason why Best Buy publishes it based on how it is written. Amazon has a broad range of digital provision with APIs categorized by main service domain and the purpose of this is well understood from a business angle.
Apart from eCommerce industry, we can also highlight some cases such as business-oriented public APIs from branded companies like Lyft and Amazon; Google Adsense API which is important for developers whose customers create their personal content through blogging, forum, wikilinks, and web publishing; and Twilio, a telephony company, which offers a basic API and markup language to promote business through a scalable, dependent and innovative voice and SMS communication channels. It is imperative to consider the shape and important features of these APIs because they reveal the potentials and vision of the business.
Design
Design outside-in implies that APIs are considered first to be deployed as a commercial provision to third party applications. For example, if you are to write a microservice API for a large ecommerce player with a huge warehouse resource application, you would write the API in such a way that the intricacy of the primary mainframe repository is eliminated by the API and of course, it also provides a basic and neat layer of objects for taking care of stock inventory, units, shelf site and so on. This is done precisely as you would have done if you offer the API as a merchant product to a consumer. This does not inspire developers to create a neat, elementary design alone but it also allows a tactical effort of substantiating novel products and services which were previously regarded as internal support function. For instance, a retailer with an extensive warehouse and logistics solution, which were once used for meeting the demands of its own stores, may abruptly decide to deliver the services of warehouse management and logistics solution to other companies.
Having a basic, neat and well-organized APIs makes this process feasible because the new group of satisfied customers can swiftly assimilate the current systems into the mainframe backend. Additionally, imposing a fee on customers for the use of the modern warehouse becomes a negligible issue because the API design is integrated with a billing and accounts management system from the outset. The operation of API versioning implies that the applicability of modern API can be deployed progressively without meddling in the affairs of customers on the current APIs. The design of APIs has a great influence on the digital value chain. If the API is workable, it is easy to achieve the mapped out goals such as sales target, cost reduction, marketability, and customer satisfaction. A workable or flexible API is always designed from the outside-in perspective.
The most crucial choices for API design are REST and SOAP. REST and JSON are the most ideal because they are adapted to mobile and web applications coupled with having a lightweight structure in contrast to SOAP and XML. For example, a forwarding agent may offer an API so that customers’ staffs can search for the location of a shipment, sending data back to a sensor. It is possible for an API to be deigned as REST APIs because these APIs are often expended by mobile or web.
In the outside-in perspective, the API design is configured according to the customers’ needs or inclinations rather the data framework. Product managers should be acquainted with the conception of outside-in perspective. If an organization has a product owner, it is better to decide the source of functionality needed from the API. This should be the driving force of your design.
The best way to understand the needs of your customers is to make enquiries. Take a look at their apps, ask for reviews, and ascertain how your APIs can restructure and enhance the lives of consumers. Create mockup API responses that provide the precise data needed. The source of data in your system is not important; pretend as if you are designing the system from a starting point.
Development
The development process targets two viewpoints; API Developers that produce API proxies and API products and Application Developers that will consume the API products. Both types of developers require an outside-in perspective.
Many APIs function as extra solution to a current application in a bid to resolve a short-term problem, such as building a mobile application. However, APIs designed from the ground-up as a separate product enable the development of not just one type of application, but any number of applications that span a variety of devices and situations: third-party software, internal applications, desktop apps, mobile devices, and the emerging world of IoT.
Unlike past trends that market to business leaders, APIs market directly to developers. And developers demand more than just a marketing website that lists the features in bullet points. They demand an intuitive, easy-to-use API that can quickly be integrated into their next project.
This requirement, called “developer experience,” is transforming the way that APIs are designed. Organizations can no longer bolt on an API as an afterthought and force developers to use it. Instead, entire markets are emerging where APIs compete for the attention of developers.
With the implementation of outside-in perspective, APIs have a tendency to reflect two things: organizational structure and database structure. Neither will produce a great API design, and both of these situations can be prevented. Taking an outside-in approach to API design means looking at your API from the viewpoint of an external developer that doesn’t have access to your code, your database design, or your organizational structure.
When an organizational structure leaks into the API, it announces to developers that the API isn’t a product; it is the output of a specific team within the organization. The result is inconsistent design and duplication or gaps in functionality.
Organizational structure creeps into an API for two reasons: 1) APIs are designed in isolation from other teams, or 2) APIs are designed around internal systems rather than external needs.
To avoid this, consider establishing an API governance board that encourages consistency across teams. A governance board isn’t meant to be an oppressive committee that stalls innovation. Rather, they represent the best interests of developers, your APIs, and your business. Great governance boards often act as internal consultants for your APIs to make them successful.
Database structure creeps in through the use of tools and frameworks that promise rapid API development at the expense of a thoughtful API design. They externalize the data through generated APIs but make one huge assumption that becomes the enemy of great API design: external developers want to use your API like you access your database.
Developers using your API don’t care how you store your data as long as it is stored correctly and reliably. Taking an outside-in design approach will prevent the database design from creeping in by encouraging you to see your API as external developers will see it.
It is essential to build an understanding across the organization of the value that “outside-in” brings to developers. This is particularly important for business unit stakeholders who need to understand how an incremental investment now can yield rewards later. This is known as API evangelism.
Testing
In all cases, testing serves one purpose; to ensure software is delivered free of defects. Users who encounter these defects become discouraged, and will quickly search for alternative solutions that meet their expectations. Even internal systems that need to communicate with each other could potentially lead to millions of dollars in losses for a company. Testing is a critical step in delivering APIs.
API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security. Since APIs lack a GUI, API testing is performed at the message layer and can validate application logic very quickly and effectively.
API testing is critical for automation testing and CI/CD process because it can coop with short release cycles and frequent changes especially the presentation layer without breaking the test outputs. API testing also requires less maintenance effort compare to UI automation testing which makes it a preferred choice for Agile and DevOps teams.
Outside In TDD lends itself well to having a definable route through the system from the very start, even if some parts are initially hardcoded. The technique puts a focus on test-driven-development, but instead of the traditional approach starts at the acceptance test level. The first test for a feature is an acceptance test and only then the feature is implemented from the outside classes first (UI and controllers), towards the inner classes (model, infrastructure). Mocks are used to describe roles of collaborators when starting to write tests for the outside classes. When a class is tested, the roles described by interfaces are implemented and the testing cycle starts again with mocks for the collaborators. Outside-In testing leads to interfaces that are written from what is useful for the client object using them, in contrast to objects that are composed of collaborators that already exist. The tests are based upon user-requested scenarios, and entities are wired together from the beginning. This allows a fluent API to emerge and integration is proved from the start of development.
By focusing on a complete flow through the system from the start, knowledge of how different parts of the system interact with each other is required. As entities emerge, they are mocked or stubbed out, which allows their detail to be deferred until later. This approach means the developer needs to know how to test interactions up front, either through a mocking framework or by writing their own test doubles. The developer will then loop back, providing the real implementation of the mocked or stubbed entities through new unit tests.
These types of tests can be performed on your API:
Performance-oriented: These are tests that make individual calls to each and every method your API provides. If a response takes longer than a specified time limit, it means that there’s a problem on that specific endpoint.
Functional testing: This type of testing works by making singular calls to each API method to thoroughly test every API function, using different kinds of payloads, or even sending data that will produce errors. Responses are then compared to the expected behavior to locate errors.
Use case testing: This is a more sophisticated type of test and can be achieved by combining calls to different endpoints into a single test. Each test should expect a specific response and execution time limit.
Deployment
Deploying APIs is still something that can be very technical, requiring a dynamic understanding of what an API is, how to construct a RESTful framework, and deploy the APIs underlying architecture. However with the growing demand for APIs and the potential for new service providers entering the space, we are seeing more tools and services available for assisting providers in deployment of web APIs in 2013. Believe it or not, even your deployment process has an outside-in perspective. The promotion of your code through various environments not only helps a team to deploy code quickly, but it can also help users engage with your code sooner, and discover value in your developer program.
The API deployment model, that is, where and how APIs are deployed, executed, and accessed by consumers, is often referred to as “microservices” – decomposing the business workflow into a set of extremely fine grained services, each of which only does one thing and does it well. A microservice is usually not greater than- lines of code, outside of which it is time to split it into two separate services. Here, the quick witted reader would ask: doesn't that sound exactly like SOA? Conceptually yes, but microservices are different in that they are tied to a set of practical deployment principles, which SOA would never do in its effort to remain agnostic of any particular type of technology. Some of the key deployment principles are design outside-in, lightweight tooling, test automation, self-documentation, programmable deployment, and multi-channel. Lightweight tooling refers to the use of simple, lightweight development tools and runtime platform. Whereas SOA in practice was open promoted through the use of monolithic middleware platforms, typically based on Java or .NET, with an integrated development environment bolted on top, APIs have emerged out of much the need for “hacker friendly” command line utilities and text editors.
Programmable infrastructure is the application of DevOps practices to deploy and operate API services and infrastructure. DevOps is a new concept, which applies agile development techniques to systems and network operations. For instance, instead of splitting development and operations into two separate teams or departments, DevOps stresses the need for operations people to work as part of a development team in order to increase collaboration. This is particularly useful for firms where APIs evolve rapidly and new versions are deployed weekly or even daily. Self-scalability refers to the ability of microservices to automatically scale out on the fly without the need for waiting for IT to provision an entirely new virtual server, allocate storage, installing the operating system. etc. Since microservices are often associated with lightweight container technology (such as Docker), it is possible for an instance of a microservice to automatically spawn hundreds or thousands of new instances of itself on the fly during peak demands (and scale down as well). Multi-channel is the principle of designing all APIs for consumption by multiple devices and end users, providing a seamless experience regardless of the client. A commonly used technique for developing reusable APIs for multiple channels is using standards from mobile development such as JSON4 (Javascript Object Notation) for messages and OAuth5 for security, which ties well in with mobile native mobile/tablet application and web based applications. A developer portal focuses on quick feedback loops, demonstrate value to your developers and offer a self-service environment to immediately engage with your APIs. This is the very definition of an outside-in perspective. Embracing modern DevOps-centric processes and tooling is critical to reducing mean time-to-production, and this should apply to your application building blocks as well. Once the application building block has been assembled and tested, deployment should be as easy as the click of a button.
Management
Management is the support given to users after new code or APIs have been published publicly. API management is the practice an organization implements to manage the APIs they expose. This is done either internally or externally to ensure that their APIs are consumable, secure, and available to consumers in conditions agreed upon in the APIs terms of use. In other words, API Management is the set of processes that enables a business to have control over and visibility into the APIs that connect applications and data across the enterprise and across clouds. API management solutions generally include the following capabilities to help a business make the most out of their API program: a developer portal, an API gateway, API lifecycle management, analytics capabilities and support for API monetization.
As we discussed in the features section above, the key to effective API management is having an inventory of an organization’s APIs that allows API consumers to digest the characteristics of the APIs available, namely:
Features: Describes what the API is designed to achieve in a form human beings can digest;
Structures: A schematic description of the APIs, including URIs, data structures, security, etc.;
Capabilities: What is the peak load the API can handle (both projected and actual) and the performance pinch points;
Sensitivities: Does the API consume or expose any data that may be subject to regulatory or privacy constraint, such as payment card data, personally identifiable data, and so on.
The API registry provides this capability, holding data on behalf of the API Gateway and Developer Portal to provide a catalog of information for human beings to digest. The registry will also help an organization manage the lifecycle of an API, cataloging the supported versions and their promotion or retirement from the organization’s API inventory.
Going back to the point about logical vs. physical architectures, the API registry is the component that is likely to be manifested within other components in a physical architecture (most likely the Developer Portal), or indeed in many organizations may be federated across both API management and other systems in an organization’s stack. However, an idealist view of the API Registry is that this is a discrete component, specifically implemented to harbor an organization’s API knowledge.
Three main requirements drive API Management solutions
API Security – Organizations must ensure that published APIs do not pose a security liability. Uncontrolled access to sensitive information from data, content and services cannot be allowed, and the underlying systems must be protected with authentication, rate limiting and quotas.
API Governance – Central governance of the APIs, reducing a lot of the technical debt and double-work. API discoverability, API reusability, API lifecycle, rich API documentation, API monetization. Portal capability, connecting API Developers to API Consumers.
API Analytics – Centralized gathering and analysis of API Analytics, providing real time dashboards. Understand API usage and performance through interactive reporting, for both API Developers and API Consumers. The digital ecosystem is evolving in many directions. Organizations are adopting multiple channels to drive newer sales channels, trigger new business models and generate more and more revenue. This triggers the need of unlocking business assets to the outside world in a secure manner. The increasing demand from Internet Business Models, IoT, social media and Cloud Adoption will exponentially increase the need to expose the business assets to the outside world by means of APIs.