Clustering Graphs and Networks

Choosing a suitable clustering algorithm for your data

Detecting graph elements with “similar” properties is of great importance, especially in large networks, where it is crucial to identify specific patterns or structures quickly. The process of grouping together elements/entities that appear based on some similarity measure to be closer to each other (than from the remaining elements) is called cluster analysis.

The similarity measure is usually calculated based on topological criteria, e.g., the graph structure, or on the location of the nodes, or other characteristics, e.g., specific properties of the graph elements. Nodes that based on this similarity value are considered to be similar are grouped into the so-called clusters. In other words, each cluster contains elements that share common properties and characteristics. The collection of all the clusters composes a clustering.

Graph clustering

Cluster analysis has many applications fields, such as data analysis, bioinformatics, biology, big data, business, medicine. However, each of these applications may consider differently the "clustering notion," which is reflected by the fact that there exist many clustering algorithms or clustering models.

How to Automatically Cluster Elements in Graphs

Even though the implementation of a clustering algorithm does not require a highly experienced developer, it is often the case that only one algorithm is not sufficient for the data analysis, or the appropriate clustering algorithm is not known beforehand. In such cases, the user seeks complete tools and solutions that offer different clustering algorithms that can apply to their graphs. Many users also have more expectations from these tools. For example, they would also like to obtain a specific arrangement of the graph elements that also mirrors this clustering information. As additional functionality, a user may also require to render the clusters with a different style or even interact with the resulting diagram and manually modify the clustering.

All the above can create limitations to users that utilize general tools providing specific clustering algorithms. yFiles is a commercial programming library that offers several ready-to-use clustering algorithms. It also allows the user to develop additional clustering algorithms and easily integrate them into any application built with the library. Except for the clustering algorithms, it also provides all the means for visualizing the graph structure and extended interaction support.

Clustering Based on Topology

yFiles offers two clustering algorithms based on graph topology that can be customizable by the user.

Edge Betweenness Clustering

Edge Betweenness clustering detects clusters in a graph network by progressively removing the edge with the highest betweenness centrality from the graph. Betweenness centrality measures how often a node/edge lies on the shortest path between each pair of nodes in the diagram. The method stops when there are no more edges to remove or if the algorithm has reached the requested maximum number of clusters. The resulting clustering is one that achieved the best quality during the process.

Edge Betweeness clustering
An example of edge betweenness clustering algorithm

Biconnected Components Clustering

Biconnected components clustering detects clusters by analyzing the biconnected components of the graph. A biconnected component is a connected component with the property that the removal of any node keeps the component connected. Nodes of the same biconnected component form a cluster. If a node belongs to multiple biconnected components, the algorithm only assigns it to one of these clusters.

Biconnected Components Clustering
An example of the biconnected components clustering algorithm

Clustering Based on Location

yFiles also offers two clustering algorithms that determine the clusters based on the current positions of the nodes.

k-Means Clustering

k-Means clustering algorithm partitions the graph into k clusters based on the location of the nodes such that their distance from the cluster’s mean (centroid) is minimum. The distance is defined using various metrics as euclidean distance, euclidean-squared distance, manhattan distance, or Chebyshev distance.

Users can also implement a Voronoi diagram to visualize the result of this algorithm. A Voronoi diagram is a partitioning of the plane that contains n points (called sites or generators) into convex polygon regions such that each such region contains exactly one of these points, and each point of the polygon is closer to the generator point that from any other point.

k-Means Clustering
A combination of k-Means clustering and Voronoi diagrams

Hierarchic Clustering

Hierarchic clustering partitions the graph into a hierarchy of clusters. There exist two different strategies for hierarchical clustering, namely the agglomerative and the divisive. The agglomerative strategy applies a bottom-up approach according to which, in the beginning, each node defines a separate cluster. At each step, the algorithm merges pairs of clusters while moving up the hierarchy. The algorithm continues until all nodes belong to the same cluster. On the contrary, based on the divisive strategy, in the beginning, all nodes are grouped into one cluster. At each step, the algorithm splits the largest cluster while moving down to the hierarchy.

In both approaches, the dissimilarity between clusters is determined based on the given linkage criterion and an appropriate distance metric as euclidean distance, euclidean-squared distance, manhattan distance, or Chebyshev distance. The result is a dendrogram which can be cut based on a given cut-off value.

Hierarchical clustering
An example of hierarchical clustering with a cut-off value and the resulting dendrogram

Example and Source Code

The clustering algorithms are part of yFiles on all platforms. yFiles for HTML comes with a Graph Clustering Sample Application. This application allows the user to experiment with different clustering algorithms and explore their configuration. Users can explore sample graphs or create their own.

Graph Clustering Sample Application}

The source code of the Graph Clustering Sample Application is available on the yWorks Github repository and part of the yFiles for HTML package.

yFiles Clustering Algorithms in Your Own Application

Test the yFiles clustering algorithms with a 60-day, fully-functional trial package of yFiles. The clustering algorithms work on the standard yFiles graph model and can be used in any yFiles-based project. Calculating a clustering is done like running other yFiles graph analysis algorithms and requires only a few lines of code.

Why yFiles?

Most complete solution

Since 2000, yWorks is dedicated to the creation of professional graph and diagramming software libraries. yWorks enables clients to realize even the most sophisticated visualization requirements to help them gain insights into their connected data. The yFiles family of software programming libraries is the most advanced and complete solution available on the market, supporting the broadest range of platforms, integrations, input methods, data sources, backends, IDEs, and programming languages.

Perfect match for all use-cases

yFiles not only lets you create your own customized applications but integrates well with your existing solutions and dashboards on the desktop, on mobile, and on the web. Developers can use concise, rich, complete APIs to create fresh, new applications and user experiences that match your corporate identity and exactly fit your specific use-cases. Browse and choose from hundreds of source code demos and integrations to get ideas and get started in no time.

Honest, simple licensing

yFiles enables white-label integrations into your applications, with royalty-free and perpetual licensing. There are no third party code dependencies.

Industry-leading automatic layouts

yFiles has got you covered with a complete set of fully configurable, extensible automatic layout algorithms, that not merely render the elements on the screen but help users understand their data and the relationships just by looking at the diagrams.

Unmatched customizability

Decades of work went into the creation of the most flexible, extensible, and easy to use diagramming APIs that are available on the market. Everything may be customized with yFiles: data acquisition and import, graph creation, display, interaction, animation, layout, export, printing, and third party service connectivity.

Algorithms included

With yFiles, you can analyze your graphs, connected data, and networks both on the fly and interactively with a complete set of efficient graph algorithm implementations. Calculate centrality measures, perform automatic clustering, calculate flows, run reachability algorithms, find paths, cycles, and dependencies. For the best user experience, use the results to drive the visualization, interactivity, and layout.

Unequaled developer productivity

Developers quickly create sophisticated diagramming applications with yFiles. The extensive API has been carefully designed and thoroughly documented. There are developers’ guides, source code tutorials, getting started videos, and fully documented source code demo applications, that help to realize even the most advanced features. Inline API documentation lookup for all major IDEs with hundreds of code snippets and linked related topics make writing robust code a breeze. Integration samples for many major third party systems help in getting productive, quickly.

Not just a static viewer

With yFiles, you can do more than just analyze and view your data. Create interactive, deeply integrated apps that don’t just let you consume data sources, but also enable users to create, modify, and work with both existing and changing data. Integrate with third party services to automatically trigger actions and apply updates. With yFiles, there are no limits: you decide what your app can do.

High-performance implementations

While it is recommended not to overwhelm the end-user with overly complex graph visualizations, of course, all aspects of the library have been prepared to work with large amounts of data. Developers can create both high-quality diagram visualizations and rich user-interactions, as well as configure algorithms and visualizations to perform great for even the largest graphs and networks.

Generic data acquisition

You don’t need to let your users create the diagrams from scratch or use a particular file format. yFiles enables you to import graphs from any data source which is accessible via an API. Programmatically build the in-memory model using an intuitive, powerful API. Update the diagram live in response to external events and changes.

World-class support

Get the best support for your development teams. Directly connect with more than a dozen core yFiles library developers to get answers to your questions. If you don’t have the time to do the implementation or your team is not large enough to do the implementation, let yWorks help you with consultancy and project work to get your team and apps up running, quickly.

Proven solution

Customers from all industries all over the world have been using yFiles for almost twenty years for both internal and customer-facing applications and tools. See the references for a non-conclusive list.

Frequently Asked Questions

What is yFiles?

yFiles is a software library that supports visualizing, editing, and analyzing graphs and graph-like diagrams. It is not a ready-to-use application or graph editor. Instead, it provides a component for graph visualization, graph editor features, and an extensive set of algorithms for automatic data arrangement and graph analysis. Software developers can use yFiles to display, edit, and analyze diagrams in their own applications. yFiles is available for many platforms.

Which platforms does yFiles support?

Right now, yFiles supports HTML / JavaScript, Java (Swing), JavaFX, .NET (WinForms), and WPF.

What kind of applications can I create with yFiles?

Developers can use concise, rich, complete APIs to create fresh, new applications, and user-experiences that match your corporate identity and exactly fit your specific use-cases. yFiles enables white-label integrations into your applications, with royalty-free and perpetual licensing. Any application that works with or displays relational data in the form of graphs, diagrams, and networks can be built with the help of yFiles.

What devices can I target with yFiles?

yFiles not only lets you create your own customized applications but integrates well with your existing solutions and dashboards on the desktop, mobile, and the web. There are versions of yFiles available for all major platforms and frameworks.

How extensive is the graph API of yFiles?

yFiles offers the most extensive graph layout, visualization, and analysis APIs available commercially. In total, there are around ten thousand public API members (classes, properties, methods, interfaces, enumerations). yFiles uses a clean, consistent, mostly object-oriented architecture that enables users to customize and (re-) use the available functionality to a great extent. API components can be (re-)combined, extended, configured, reused, and modified to a very high degree. It is not mandatory to know the complete API, of course. Most applications only require a minimal subset of the full functionality, and the advanced functionality and APIs may only be required for implementing unique requirements.

As a developer, what can I expect from yFiles?

yFiles helps developers quickly create sophisticated diagramming applications. The extensive API has been carefully designed and thoroughly documented. There are developers’ guides, source code tutorials, and fully documented complete source code demo applications that help to realize even the most advanced features. Inline API documentation lookup for all major IDEs with hundreds of code snippets and linked related topics help in writing robust code, efficiently. Integration samples for many major third party systems help in getting productive, quickly.

Is yFiles Free?

No. yFiles is a commercial software library. If you decide to use yFiles in your application, you’ll have to pay a one-time fee. You also have the option to subscribe annually for technical support and updates.

How does the licensing work for yFiles?

yFiles enables white-label integrations into your applications, with royalty-free and perpetual licensing. There are no third party code dependencies. Licensing basically works on a per developer basis. Please refer to the pricing information and software license agreements of the respective product for more details.

What kind of support can I get for yFiles?

The yFiles libraries come with fully documented demo applications, detailed API documentation, and extensive developers’ guides. Apart from that, yWorks also offers professional support services for your development teams. They can connect directly with more than a dozen core yFiles library developers to get answers to their programming questions. Optionally, if you don’t have the time or necessary team, yWorks can help you with consultancy and project work to get you and your apps up running quickly.

How is the release cycle for yFiles?

There is no public roadmap for yFiles. yFiles usually gets a new major feature release about every 10 to 15 months, with bugfixes or minor maintenance releases in between as required. Typically there are between one and five bugfix releases for each major release, and previous releases get important bugfixes, too. yWorks tries very hard to keep the libraries and APIs backward compatible so that customers can update to the newest version of yFiles regularly with little to no effort and still benefit from performance improvements and new features.

Can I edit my graphs with yFiles?

With yFiles, you can do more than just analyze and view your data. You can have interactive, deeply integrated apps that don’t just let you consume data sources but also enable users to create from scratch, modify, and work with both existing and changing data. Integrate with third party services to automatically trigger actions and apply updates in real-time and publish changes to third party systems while the user works with the graph. It’s up to you to decide what your app can do.

What kind of layouts does yFiles support?

yFiles comes with the most extensive set of fully configurable, extensible automatic layout algorithms, that not merely render the elements on the screen but help users understand their data and the relationships just by looking at the diagrams. yFiles includes hierarchic, organic (force-directed), orthogonal, tree-like, radial, balloon-like, and special purpose layouts. yFiles also supports incremental, partial, and interactive layouts, as well as various edge routing and automatic label placement algorithms.

Are the layout algorithms configurable?

Layout algorithms support various settings and constraints and are fully customizable in code. They support different node sizes, nested groups, bundled edges, orthogonally and octilinearly routed edges, consider and automatically place node, edge, and port labels. Nodes may be partitioned and clustered, and different layout styles can be mixed in the same diagram.

What kind of graph analysis does yFiles support?

yFiles lets you analyze your graphs, connected data, and networks both on the fly and interactively with a complete set of efficient graph algorithm implementations. Choose from a range of different centrality measure implementations, automatic clustering algorithms, network flow algorithms, reachability and connectivity algorithms, pathfinding variants, cycle, and dependency analysis algorithms. For the best user experience, use the results to drive the visualization, interactivity, and layout.

What parts of yFiles can be customized?

yFiles has the most flexible, extensible, and easy to use diagramming APIs that are available commercially. Every aspect of the functionality is customizable with options ranging from high-level configuration settings, down to low-level implementation overrides: data acquisition, import, graph creation, display, interaction, animation, layout, export, printing, and third party service connectivity.

How can I get my data into yFiles?

End-users don’t need to create the diagrams from sketch or use a specific file format. yFiles lets you import graphs from any data source that is accessible via an API. Developers can populate the in-memory model using an intuitive, powerful API, directly connecting to their preferred data sources. Diagrams can be updated live in response to external events and changes.

How can I get my diagrams data back from yFiles?

The in-memory graph model lets you export all the information to any system and file format. There are built-in export options to various file and image formats, but as a developer, you can create your own glue code to connect to arbitrary data storage systems and third party services.

Is the diagram size limited?

Theoretically, the only limiting factor for the number of graph elements is the size of the computer’s memory. In practice, performance is also a limiting factor. For the vast majority of use-cases, yFiles delivers best-in-class performance out-of-the-box. For very large visualizations and data-sets, there are options available that let developers tune between features, running-time, and quality of the results. yFiles can deal with graphs of any size and is only bound by the memory available and the runtime complexity of the algorithms. Large graphs may require adjusting the default settings and performance depends on more than just the number of elements in the diagram, but also the structure of the graph, the algorithm and configuration, as well as platform and hardware capabilities.

Who is using yFiles, already?

Customers from almost all industries all over the planet have been using yFiles for nearly twenty years, to create both internal and customer-facing applications and tools. Clients include both single developers and the largest corporations and organizations in all of academia, public and governmental services, and of course, the commercial space. See the references for a non-conclusive list. Naturally, there are the big well-known software corporations among yWorks’ customers (unfortunately only some of them allow yWorks to list them on the references page), but there’s also a great lot of companies that are not traditionally known for software, but who still have their own IT departments create software for their intranet or customer-facing applications. And last but not least, smaller companies without IT departments that let third party implementors create useful diagramming applications with the help of yFiles for them. yFiles at its core is a generic diagramming component that is use-case agnostic and can be used to create graph and diagramming-centric applications for any business domain that requires working with or displaying connected data.

How long did it take to implement yFiles?

yFiles started as a university project at the University of Tübingen in the late 1990s. Since 2000, yWorks has taken over all development and has been working continuously with a core layout-team of two to eight developers on improving the layout algorithms. The layout algorithms alone, as of 2019, took more than sixty development years to implement. A team of more than 20 developers has been working on the implementation for the visualization and interaction and the support for the various platforms yFiles supports, totaling in more than a hundred years of development for the visualization. Porting yFiles to a new platform in the past took between three and about 15 development years. Most platform variations were implemented in between six and ten calendar months.

How long has yFiles been around?

yFiles started as a university project at the University of Tübingen in the late 1990s. The company yWorks was founded as a spin-off of the university in 2000 when the first commercial customers wanted a license for yFiles. Since then, it has been developing and improving the library. It all started as a Java library, and over time, yWorks improved and even rewrote large parts of the library to add new features and support new platforms.

Who is the company behind yFiles?

yWorks is the company behind yFiles. It was founded as a spin-off of the University of Tübingen in the year 2000 specifically for licensing and supporting yFiles commercially. The German company is a privately-held, headquartered in Tübingen. More than 25 employees are working at yWorks, about 20 of which are developers, working on yFiles and the tooling around the libraries. The library developers also provide support and implementation services to yFiles customers. So as a developer, you will get first-class, highest level support directly from the team that implements the libraries.

What does yWorks specialize in?

Since 2000, yWorks is dedicated to the creation of professional graph and diagramming software libraries. The software yWorks creates, enables customers to realize even the most sophisticated visualization requirements to help them gain insights into their connected data. Their main product is the software programming library family yFiles, which is the most sophisticated and complete solution available for diagramming applications on the market, supporting the broadest range of platforms, integrations, input methods, data sources, backends, IDEs, and programming languages. yWorks has set a track-record in providing the most extensive layout and diagramming solutions for developers on all major platforms. In addition to creating, maintaining and supporting the libraries, yWorks also provides professional consultancy services in the area of visualization and diagramming. In addition to that, yWorks also provides a set of smaller software tools, both free and commercial, end-user facing and for software developers, closed-source and open-source.

Does yWorks own all the intellectual property for yFiles?

yFiles does not depend on any third party library, except of course at runtime, where it depends on the runtime of the platform. yWorks owns the IP for all implementations in the core yFiles library. Some demos show the integration and make use of third party software, but they are not required for other cases.

Which papers and algorithms does yFiles implement?

The list of algorithms implemented by yFiles is long. For the common graph algorithms, we use the traditional implementations with the standard optimizations. For many of the layout algorithms, ideas for the implementation base on publicly available papers. Some algorithms (specifically the orthogonal layout and the balloon layout) we created and helped with the creation of the algorithms and (co-)published the papers for the algorithms. Most layout algorithms have been vastly modified, tuned, and enhanced, though, and don’t follow the original implementation ideas, anymore. yWorks added useful features to these implementations to make the algorithms work in less theoretical environments. We removed previously existing constraints of the original implementations and added new ideas to make the algorithms useful for real-world usage. For most of these changes and improvements, no papers have been published.

Can I get the papers for the layout algorithms used in yFiles?

For some of the algorithms, you will find papers that describe the core idea of the layout algorithms. For most algorithms, yWorks massively enhanced and modified the algorithms to support more advanced features that are frequently required in real-world diagrams. For these modifications, we did not publish any papers. As a commercial yFiles customer, you can obtain a license to the source code of yFiles where you can read, learn about, and modify the algorithms in documented source code form, according to the license terms.

Hide Description
Open in yEd LiveDownload