In May 2016, Ceetron released 3D Components with Python bindings. When this happened, I (as responsible for quite a lot of the binding work) was tasked with writing a blog post about Python and 3D Components. The editor of the blog bluntly told me: “Andres, just write something marketingish about our new release”, for example along the lines of ‘debunking the myth of not being able to create high-performance CAE apps with Python alone.’ I said to myself, no need for debunking here, and most CAE engineers I know have a pretty accurate understanding of the limitations of Python – you can’t win with it alone, you need compiled language performance behind it. Indeed, I never wrote that blog that our blog editor wanted me to write (and now he owes me a beer).
12 months later and having worked tightly with some pretty sharp CAE engineers with solid Python skills on their Python projects, I decided to revisit the topic. This time I decided to write my own story, about my (and the CAE community’s) love in Python for small-scale as well as serious-scale coding. I also decided to talk with Fredrik Viken, our CTO, about Ceetron’s product strategy and how Ceetron wants to contribute to the use of Python in the CAE community, and do an informal interview with Tim Hunter, Ph.D. and President of Wolf Star Technologies (provider of CAE software written with Python, customer of Ceetron, see http://www.wolfstartech.com/). But I started out by going all the way back to the 80s, when it all started, to reflect on the evolvement of some industry issues that have had profound impact on the pick-up of Python in the community:
Clearly, from quite simple decision-making helpers to true design tools, capable of accurate predictions, the importance of simulation software for engineering has immensely increased over the last 30 years. The simultaneous development of numerical methods, physical models and computational power has given these tools a strategic role in such important fields as aeronautics and automotive: not a single part is designed or even formed without the help of a computer nowadays.
The emergence of CAE software market took place in the 80’s, and at the time its structure was very different from what it is today. Initially, professional CAE software edition would most commonly stem from either startups driven by numerical scientists or behind the scenes, in the form of in-house projects of industrial corporations. In both cases, in those early days, the offerings were based on the “solver”, the component that transforms input data into predictive results. And that only seems normal, as the solver is the core of this business.
Given the hardware limitations of the day, the challenge of computing quickly and accurately were immense and most resources were devoted to that goal. There was little left for fancy or user-friendly user interfaces, not to mention programming interfaces for result exploitation or management by end-users.
After a decade or so of consolidation, during which the most intrepid scientist startups became solid software editors and most corporations dropped in-house core development altogether, software packages had evolved from solver-centric to extended offertings, including top-notch user interfaces and a variety of useful side features: extensive databases, reporting facilities, and the first programming interfaces that allowed to further process results.
At the turn of the century, these very complete and mature offerings attracted new actors – actually fish, and very big ones -: CAD software editors. Initially focused on the design of objects, these companies had slowly extended their offering to the computation of the use properties of these same objects. Given their position in the product life-cycle (at the very start of it) and their greater financial power, this companies went into an acquisition frenzy of small- to medium-size simulation editors (Autodesk has for example completed around 50 acquisition since 1992, including Moldflow and Blue Ridge Numerics; recent examples of acquisitions by Siemens PLM includes LMS, CD-Adapco and Mentor Graphics). Some of these editors did survive however, using the same acquisition strategy on their kin to become the large editors we know today (incl. for example ESI and Altair) and escape the appetite of the CAD editors.
Finally, after these large-scale changes on a technologically static market – no real game-changer had taken place since parallel computation made its way in the 90s – the 2010s suddenly became an exciting decade to live. Very new offerings started appearing everywhere: new because of the production mode of their core solvers (open source), new because of the proposed infrastructure (the famous Cloud) and finally new by their licensing scheme (SaaS). As in many other fields – but with a noticeable lag – the established actors of the CAE market suddenly saw (hmm, not “saw”… “are seeing”, it’s still going on) these new kids on the block set new rules, and more popular ones given the modern technological environment we live in (e.g., SimScale, OneShape and Penguin Computing).
In a few paragraphs, this summarizes as I see it (and have lived it) the emergence, growth and current situation of the CAE editor market. Of course, the pattern is a coarse one and many exceptions can be found along the way, but very broadly speaking we have seen solvers developed small teams of scientists integrate into very complex systems such as those marketed by CAD corporations.
Where does this leave the CAE engineer? In a mess, is my answer. One can imagine that in the 80s, when the software was simple and cross-solver studies were eccentric, the learning curve had a gentle slope. Today, it’s a very different ball game: the systems have become very complex and full studies often involve using multiple software packages (the results of one being fed more or less directly to the next). The learning curve suddenly got steeper – from a nice stroll in the woods, we are talking about the Himalayas nowadays
Pre-processing a simulation (setting up its data) is very close to the real-life application and to the trade of the end-user, by its trade slang and its specificities. It seems hard to generalize this step, and I am afraid an engineer has no other choice but to master the different interfaces.
Post-processing is different. This is the step where solver results are analyzed (inspected) and … processed (require some user-defined calculation to make sense). I am convinced that here there is room for a generic and more efficient approach.
Even if some of the operations involved in result analysis depend on the end-user’s real-life application, most tools that are necessary to analyze fields on a mesh are generic: cutplanes, isosurfaces, sensors, plot generation, … There is no need to learn to use different GUI’s and it seems to me that there is room for substantial efficiency gains if you are doing so today. A unique tool to analyze results from different solvers is the way to go: unify your workflow and spend more time using the software than learning how to use it.
When it comes to result processing, the CAE professional would enjoy to seamlessly read results from one software package, process them and maybe insert them as data to another package. Depending on the nature of the processing, the training and the likes of the engineer, these operations are today performed using a wide variety of tools and technologies: Excel sheets (and maybe VBA macros), compiled languages (C++, Fortran, C, …), scripting languages (DOS, VB, Linux shells, …).
In comes Python. This is back in 1994 (when version 1.0 was released, though implementation started in 1989). The creator, Guido van Rossum, an accomplished software developer who later went on to win several professional prizes (including the 2001 Award for the Advancement of Free Software from the Free Software Foundation, the NLUUG Award in May 2003, and recognition as a Distinguished Engineer by ACM in 2006) wrote the following about its creation: “in December 1989, I was looking for a “hobby” programming project that would keep me occupied during the week around Christmas. My office … would be closed, but I had a home computer, and not much else on my hands. I decided to write an interpreter for the new scripting language I had been thinking about lately: a descendant of ABC that would appeal to Unix/C hackers. I chose Python as a working title for the project, being in a slightly irreverent mood (and a big fan of Monty Python’s Flying Circus).” What came out of this process was Python: a standard and powerful scripting languages that allows to work with complex data structures coded in other languages, and to perform operations on them within a simple text editor.
Today, widely taught as a beginner’s language in engineering schools, it is extensively used by CAE engineers (it is probably the most popular language for CAE end users), and is according to IEEE also the third most popular programming language, after C and Java, but before C++ (source: http://spectrum.ieee.org/computing/software/the-2016-top-programming-languages).
Not only does Python allow one to produce very clean and legible source code, it also has the very interesting feature of being a reference. Whatever the need, there are modules for everything only a few clicks away. For example, NumPy and SciPy for scientific computing, or MatPlotLib to generate plots. Ever since Python started out in the early 90s, it’s community has grown steadily and the amount of available functionality has followed. It’s the success of standardization in terms of programming.
Therefore, it’s not surprising to observe that a growing number of simulation editors nowadays offer Python APIs, which allow user to access results (or even the GUI for customization purposes) [Abaqus][Ansys][MSC Mentat]. I would expect this movement to expand in the coming years.
The true problem for the CAE professional remains: how to make his or her workflow more efficient in a multi-software package environment. On one hand, graphical user interfaces to analyze results are all different, and on the other hand, the application programming interfaces (APIs) to process results will never be the same (if any are provided). There are a lot of interfaces to master – graphical or programming – to become fully operational across a complete workflow. The idea of using a unified general and unifying framework is not far, a tool capable of reading a wide range of results files and expose them to the user in a unified way, both for results analysis and result processing in Python.
It is this vision of unified platform for the CAE community’s post-processing needs that I and my colleagues in Ceetron decided to take a serious stab at in 2015 and 2016. A Python version of 3D Components was made available in May 2016 as part of the 3D Components 1.4.0 release. It is essentially a development framework that covers the opening of a result database from virtually any software package (we support most industry-standard formats), the building of high-quality 3D visualizations (whether on desktop or in the Cloud), and the creation of custom post-processing tools that will lead them safely through their workflows. We are currently working on extending its post-processing features for the specific trades of our end-users and developers. Said Fredrik Viken, CTO in Ceetron: “We wanted from the outset to solve, through a unified, cross-solver, end-to-end workflow platform, what we saw as an industry-wide problem for the CAE community, one that tens or hundreds of thousands CAE engineers face on an everyday basis. And of course, as providers of professional software for building software, we have deep respect for the open source work that Python Software Foundation is undertaking. We are happy to add a brick to its overwhelming status of industry standard. Moving forward, Ceetron plans to continue the Python support in C3DC and we are also evaluating adding a Python based API to our upcoming 3D Viewer Pro. This will add an even faster time to market for users who want a head start on their post-processor/viewer. They will start out with a full post-processor/viewer, and be able to customize the UI and add their own trade logic and custom features using Python.”
But I promised you some feedback from Tim Hunter, Ph.D., President of Wolf Star Technologies: “We started out this joint project on True-Load in spring 2016, after a thorough evaluation of alternatives. What has impressed me most in this project is the ease of direct access to key developer resources in Ceetron, and the design in 3D Components of APIs with exactly the right abstraction level for a Python programmer. Ceetron has a rich set of resources for direct database access and the sky is the limit with visualization options using the Ceetron 3D Components. Furthermore, the integration with standard GUI interfaces like PyQt makes development very straight forward, modular and intuitive. And of course, I share [Andres’] enthusiasm for Python for post-processing applications, that’s why we chose to develop everything that we do in Python, in the first place.”
We will try to provide an in-depth and much more technical interview with Tim in a follow-up blog post.
Feel free to contact me or one of my colleagues in Ceetron if you want to learn more about our SDKs for Python programming.