Grid Computing and Super Computing
By Michael Osias, IBM
Grid Computing and Super Computing
Grid computing is distributed computing over the network enabled
by open standards. It is the next evolutionary step in
distributed computing resulting from the availability and convergence
of capabilities offered by other distributed technology such as Common
Object Request Broker Architecture (CORBA), distributed
computing environment (DCE), Java 2 Enterprise Edition (J2EE), Web services, and
the build out of network capacity during the late 90s. The increasing
computing demands and requirements that governments, business, and science
and research are placing on information technology are driving the
convergence of these distributed technologies to address these demands.
Initially, Grid computing was formed in the scientific and
research communities to enable large scale computations whose
capacity requirements exceeded any single organization's resources.
Collaborating researchers needed a way to
aggregate and share super computing resources and make them available to do
their work. In essence, by linking these resources together using Grid
resource sharing protocols, a larger, virtual
super computer was formed composed of the shared resources. Often, these
shared resources were comprised of heterogeneous compute and
storage resources and tended to be geographically distributed. Increasingly,
industry began to face the same pressures and needs of having to handle
increased workload and data storage requirements, often within or
between distributed enterprises, without additional capacity.
As these types of large scale shared resource complexes became
more commonplace, efforts to standardize the infrastructure and services
began. Early leaders of the standardization efforts were from research,
science, and academia. Industry and governments, searching for solutions
to similar problems, recognized the emerging Grid technology as a
viable computing technology to solve problems today and provide a
clear evolutionary path to the future. As adoption grew, so did the
participation in the development of the standards, which today include participants
from industry, government, and academia. As a result of this expanded
adoption and participation, there is a rich set of Grid computing standards
and technology.
Industry leaders are heavily involved in developing and
refining standards. IBM in particular has played a significant role in the development
of the standards and contributed to the development of the open
source standards based software, the Globus Toolkit. IBM has incorporated
Grid technology across all lines of business including services, hardware,
and software, and is delivering over 100 leading edge production Grid
projects around the world.
Grids of a Feather:
The Global Information Grid and Grid Computing
Grid computing is a parallel and distributed architecture that enables
the sharing, selection, and aggregation of geographically distributed
resources. Specific kinds of resources applicable to the Department of Defense
(DoD) environment include computers (PCs, workstations, clusters,
supercomputers, laptops, notebooks, mobile
devices, PDAs, etc), software (e.g., Service Providers vending content
and expensive special purpose applications on demand), catalogued data
and databases (e.g. transparent access to unit readiness, weapons
characteristics databases etc.), special devices
and sensors (e.g., reconnaissance satellites, UAVs, etc.). Additionally,
Grid technology addresses resource availability, capability, costs, and
user Quality of Service (QoS) requirements. Grid infrastructure services, such
as schedulers, resource managers, information services, and data
services, are all essential to the Grid complex to provide inherent resiliency,
resource optimization, and non-trivial quality of service while managing resources
in the battlespace or DoD environment.
Traditionally, as a reflection of the origin of Grid technology, the
usage model is oriented toward efficient use of all available processing
horsepower. Grid standards are now leveraging innovations in Web services
technology with Open Grid Services Architecture (OGSA), moving Grid
technology toward a service-oriented computing model. This enables market-based utility computing and a new
Net-Centric paradigm.
Transforming defense intranets into powerful, self-managing,
virtual computers is a technical objective that Grid technology can enable. This
large scale virtual computing environment, supported and enabled by a
common Grid computing substrate, enables a processing environment for full
Net-Centricity. It fits with the DoD Global Information Grid (GIG) vision,
with DoD's current state consisting of a vast collection of heterogeneous
systems, just beginning to share communications and computing resources on
wide area networks. Soon the GIG will be moving to implement the
GIG Enterprise Services (GES). These enterprise services provide
ubiquitous service-oriented access to DoD data. A Grid complex supporting these
data services means that they perform according to quality of service
policies, are standards based, and universally accessible to users and
other applications. Grid services technology allows rapid, efficient coupling
of legacy transactions systems and will enable new warrior or
business applications. In addition, Grid-enabled enterprise services will begin
to address globally distributed computing issues.
Grid Proliferation and the Global Grid
As the momentum of Grid adoption continues to grow, we will
see more domains and more Grids being built out within, and at times,
between these domains. The proliferation of Grid technology is where the
Grid standards are so important. To be able to join together Grids, whether
the intent is to tap into another organization's Grid for
collaboration and form an inter-Grid, to interface with an infrastructure provider
to augment an existing infrastructure and dynamically increase capacity
on demand, or if statewide Grids want to interconnect with other
states, universities, or federal Grids, the open Grid standards enable
Grid interconnectivity. As the technology matures and Grids continue to be
built out, inter-Grids will become more commonplace and provide
the technical and architectural substrate for the next generation of computing.
In addition to allowing organizations to share resources within
or across organizational boundaries in an inter-Grid scenario, individuals
or departments may want to make resources available to a larger
Grid. Standards again are the enabler, and using standards-based Grid
middleware infrastructure services, resources can easily be added to existing Grids
and made available for use. There are many choices for this infrastructure
software, from open source software such as the J2EE-based Globus Toolkit
to commercial offerings such as the IBM Grid Toolbox and DataSynapse.
Getting Started with Grid Services
Web services technology provides a standard approach to
separating interface from implementation details in an Internet-based
distributed computing environment. By decoupling interface from implementation, classical Grid technology can
be used to develop systems that can be composed and
reconfigured dynamically at run time. Building on Web services technologies, the
Open Grid Services Infrastructure (OGSI) specification defines Web
service interfaces and behaviors for creating, managing, and exchanging
information among stateful Web service instances. OGSI is comprised of an extension
of the base Web services standards, which currently do not address the notion
of stateful Web services instances, and the definition of a set of interfaces
that describe and standardize requestor-service interactions. These
interactions provide the basis for additional semantic interoperability not
provided in the base Web service standards. In essence, these interfaces define
the basis of a Web services programming model.
With the exception of the simplest data retrieval or computation
services, a service needs to keep some kind of state to provide any value above
and beyond a basic Web-based extensible markup language (XML)
document processor or a remote procedure call (RPC) interface. Many services have
a distinct state associated with every single client accessing them.
A common example of such a service is a shopping application with the
shopping cart unique for each client. An e-commerce business can be
created simply by integrating and coordinating sets of services which include
on demand supply and procurement, shipping and routing, and
accounting and billing, as well as Web application infrastructure services such as
a shopping cart, an online address book service, and payment processing.
By using available services, like a generic shopping cart service, the
company saves money by assembling these component services into higher level services and applications. There is
no need for them to develop and manage code and infrastructure for
basic features such as a shopping cart. The service provider that provides
the shopping cart must be able to accommodate many different
shopping carts from multiple customers. In addition to the obvious service
data elements needed for state such as the shopping cart contents and
session information, the service provider must also track activity and usage data
for billing the appropriate organization (not necessarily the person using
the shopping cart, but the e-commerce customer who has integrated
the service into their online business).
Another example is that of a database query service. It is
analogous to using a database command client, such as a DB2 command line
interface, where each client has its own connection and
consequently connection session state to the
database server. This allows concurrent access to the database server with
different clients performing a variety of database operations simultaneously. To
extend the notion of a database connection to a services oriented architecture, a
similar session and state management facility is required. Otherwise, using
the classical Web services definition and forcing an analogy of a Web service
as a singleton, the service is able to accommodate only a single data
service client at a time, typically in the call pattern of a blocking call. Using
the OGSI component programming model, each client can have its own instance
of a service by either inspecting a Grid data service registry or by invoking
the createService() port reference to instantiate a new service instance.
With this model, multiple clients can interact with the database service with
their own individual session state preserved. An implementation strategy for
this kind of service could involve exposing stateful j2ee entity and session beans
as ogsi-compliant Grid services to provide data access.
A reliable file transfer service, like the one provided with the open
source Globus Toolkit, also requires the ability to maintain instances and state if it is
to service more than a single client at any given time. The persistent
factory service again is used to create a unique instance of the service for a client.
For those familiar with software patterns, the factory pattern mentioned herein
is very similar in concept. It is an OGSI base service whose purpose is to
create instances of services, much like a static Java factory object would create
well formed instances of specific objects using getInstance(). Once a
service instance is created, the client can then interact with the service to
provide such information as source and destination, and even register
for periodic updates from the service instance. The service state
is maintained by service data elements which are stored as part of the
service instance. Each instance has its own set of service data which has
different values. An example of the service data for the reliable file transfer service
is the percentage of the file that has been transferred. A client may subscribe
to this service data element using the OGSI notification framework
to receive periodic updates about the amount of the file transferred.
By having stateful service instances, potentially transient but also with
a lifetime beyond that of the initial SOAP envelope processing, a
Grid service can provide services to many different clients, and possibly
offer different quality-of-service levels and corresponding implementations
based on the priority of the user. As an example a Grid service may
manage bandwidth allocation between two network endpoints based on the
user level and priority, allocating a higher percentage of the link to the
higher privileged user.
The OGSI programming model is designed to integrate key
Grid technologies to create a distributed system framework to provide for
the controlled management of the distributed and often long-lived state that is commonly required
in sophisticated distributed applications. Fundamental Grid technologies
include information services, resource management, data services,
and security. The creation of Grid service instances using a persistent
factory service enables the leveraging of the resources in terms of intelligent
load balancing and service routing. A createService operation on a
factory service may involve a number of steps before the service implementation
is instantiated and a Grid Service Handle (GSH) returned to the client.
For example, the factory service may intercept the instantiation of the
service implementation to determine if a suitable instance already exists. It
may also inspect the Grid infrastructure by using OGSA-managed
resource services to determine the current load on a node, or consult an
index information service to determine what suitable resources are available in
the Grid complex. Ultimately, the factory may instantiate a new
hosting environment on another node which is used to instantiate the service. A
GSH for the service instance on the new node is registered in the index
service and returned to the client. The factory service may have its own
mechanisms for these processing cases, or may hand off to underlying resource
management Grid infrastructure services such as brokers and schedulers. The
factory service then becomes more of a coordinator of workflow across
various Grid infrastructure services.
There are many resources such as IBM RedBooks and the
IBM DeveloperWorks website Grid Computing zone. Also available is
the IBM Grid Toolbox for Multiplatforms v3.0. This enhanced derivative
of Globus allows developers to easily install, configure, develop, and
manage Grid services. Installation, systems management, and
application development enhancements let developers focus on Grid
applications development and less on the Grid infrastructure itself.
About the Author
Mr. Michael J. Osias is a Technical Architect for IBM Grid
Computing Americas, with a focus on Department of Defense and other
Government agencies. He has e-Business technical architecture experience in large
e-Business infrastructures including portals, B2B, e-Commerce, and
legacy integration and transformation. Michael has a background
in telecommunications, treasury and finance operations, and
defense industry. Prior to being a Grid Technical Architect, Michael was an e-Business Architect covering
several telcom and manufacturing accounts, while conducting independent
Grid Computing research and participating in the Global Grid Forum. He
received an IBM Invention Achievement Award for patent submissions and
technical disclosures in such areas as web search technology and distributed
transaction processing. Also, he is an IBM Certified WebSphere Systems Expert. He has an educational background
in Mathematics and Computer Science with the State University of New
York and other higher education institutions.
Author Contact Information
Michael J. Osias
E-mail: mosias@us.ibm.com
Cell: (201) 230-4905
URL: http://www.ibm.com/grid
Career Planning
Those looking to become key players in Grid technology and
get started developing Grid services and Grid architectures should have
a background in J2EE and Web services technologies, as well as
distributed systems. It's a small step from there to Grid computing and Grid services.
|
|
DACS Gold Practice Initiative
ROI Dashboard
