Scientific programming or scientific computing

“When I use a word,” Humpty Dumpty said, in rather a scornful tone, “it means just what I choose it to mean—neither more nor less.” (LEWIS CARROLL)

Scientists use computers as everebody else, so what is scientific (engineering, technical) computing or programming? I’ll define scientific computing as using of some kind of the field specific libraries (modules or tools) e.g. you read data from the file call functions which process the data and plot the result. Actual program depend on language you use such as Python, Matlab, R, Maple etc. The point is that you need not to implement corresponding numerical algorithms by yourself, this job is already done by module (or library) author. Scientific programming starts when you can not find what you need.

Scientific and Technical Efficient Programming (STEP)

Computers have become an important part of scientific research and technical development. They are essential both for collecting and analyzing experimental data and for solving complicated mathematical problems arising from theoretical considerations. Due to modern computing environments many scientists and engineers do not any programming in compiled languages like C or Fortran at all, but if they do they should do it in efficient way.

It is hard to define exactly what is Scientific and Technical Efficient Programming (STEP) but two main features are clear.

  • It is not evident whether the result is right or wrong. It is even difficult to say what do we mean by “wrong”, it might be bug in the code or inaccuracy in the model or numerical instability, and it is much more difficult to prove that result is correct.
  • Scientific application should always be tuned to get peak performance. The more computer power we have the more accurate results we can get, more complex problem can be solved.

These characteristics make standard programming and debugging techniques described in the most of programming books and courses useless. STEP demands scientific knowledge, the proficiency in numerical analysis, computer architecture and programming languages. All attempts to screen the programmer from “gory details” are bound to failure, unless problem is very simple.

Which Language?

In my opinion only two of modern languages namely C/C++ and FORTRAN/Fortran are fit for STEP. (I see C++ as improved C and Fortran as improved FORTRAN.) None of them is better than other and both have their strong and weak points. One can write good or bad (or ugly) program in any language. From the performance point of view nothing can beat assembler.

All function that affects performance is written in assembler and C code is just used for wrapper of assembler functions or complicated functions. Also I use many inline assembler functions,unfortunately most of commercial compiler can’t handle inline assembler.
Kazushige Goto,GotoBLAS2 FAQ

C is more flexible and closer to hardware but some important constructions are absent, mainly multidimensional arrays (though it has more flexible but less efficient one – array of arrays). Working in industry I used C and C++ for numerical alculations and even rewrote some FORTRAN codes. However I recognized that most of my code deals with very rigid structures which simply reflect standard FORTRAN ones. Actually it is not surprising since most of numerical algorithms were desingned with FORTRAN in mind.

One the other hand most of FORTRAN deficiencies were corrected in Fortran. It is modern programming language, with dynamic memory allocation, standard C interoperability and a lot of other good things.

SPIPS (Eightfold) Way

Working with real-life problems you will need the following:

scientific and/or engineering knowledge;

  • numerical analysis and programming;
  • knowledge of modern computer architecture;
  • source code (to have full power for optimization and tuning);
  • experiment with different algorithms and implementations.
Analyse critically what you read or hear.
The complexity of modern machines and compilers is so great that
theoretical estimations of algorithm complexity (bread and
butter of computer scientists) may be misleading. There are too
many layers of transformation from source code to hardware,
so small changes in in the source code may change performance greatly.
The speed of innovation in hardware and compilers is so high that
well known and tuned algorithm may already become obsolete
or at least suboptimal.
See big picture!
Your customers may have only vague understanding of
computing but they are professionals in their field.
Talk to them, be sure that you understand the problem,
get as much information as possible concerning ranges
of variables, relative probability of different scenarios and all that.
Use the right instrument for the job!
More complicated instruments are not necessary better.
In many cases it is less work to write few tens of lines
in C than to install large library and tune it properly.
Don’t copy code!
  • It may be illegal.
  • It may be dangerous.
  • Even code written by a Great Master may be
    suboptimal for your specific case.

Study the code, understand it and write your own.

Don’t leave loose ends!
Fix any problem as you find it. Small bug may be a symptom
of serious flaw in algorithm design or understanding of the problem.
Don’t make quick and dirty workarounds.
Find the reason and fix the problem properly.
Don’t be overprotective!
Remember that a crash may be annoying but a wrong answer is disaster.
Don’t be a slave to the rules!
Including those above.

Industrial Physics

Make everything as simple as possible but not simpler.
(attributed to Albert Einstein)

Industrial physics is not a branch of physics, generally it means solving of practical real life problems by person, trained as physicist. Contrary to what I call “inventor’s approach”:try this, try that, keep trying until you get something working;
or engineer’s approach: this is a problem of such-and-such type hence use this-and_that method – industrial physics is based on understanding of the problem.

I am physicist by training, trade, and style of thinking … so understanding for me is building of proper physical model,not too general and not oversimplified and making mathematical description of it. Then comes algorithm design and implementation, running simulations, analysis.