Prof. Guerraoui: How Transactional Memory Influences Our Future

At one of the programming forums we were pleased to be able to speak to Rachid Guerraoui (École Polytechnique Federale de Lausanne, EPFL), whose research is devoted to concurrent and distributed programming, across systems ranging from multiprocessors to wide-area networks. This topic is of high interest to us, as Devexperts solutions for the financial industry widely use concurrent algorithms.

Q: What got you interested in distributed computing?

RG: You usually meet someone who gives a talk; I think I saw Leslie Lamport once giving a talk. I was listening to him while he was explaining mutual exclusion. I said to myself “Oh, this is interesting.” I learned it step by step, and then it was the only thing I knew so I had no choice but to continue. That’s how it works usually, and that’s how it worked for me.

Q: As we understood from your recent works, some time ago you were researching Software Transactional Memory (STM). More recently, Hardware Transactional Memory (HTM) is rapidly being developed. Is HTM going to be used for real-time tasks, especially for financial tasks?

RG: You never know with these technologies whether something is going to be used or not. You can make some expectations or predictions, but you never know. What is happening today is that all computing is becoming concurrent and distributed; every task. Especially something that is real time and liquid like financial technology (FinTech), where you have to make predictions very fast. Everything is parallelized and everything is fast. So really, concurrent computing is important. But it’s not clear whether STM or HTM is going to be the solution. Maybe it would be slightly different.

Q: About the STM vs HTM problem: STM has been available for a while, but it was never applied in practice due to low hardware efficiency. HTM was always considered a possible solution for the future, but now better hardware is appearing where HTM is already being used in small things like implementing small mutexes. Do you see some potential for HTM to be used on a large scale?

RG: The potential is really huge, as there are a lot of people using HTM or extending HTM for all kinds of applications. The potential is huge, but this does not mean that it will be adopted. Because adoption depends not only on the technology power, but also sometimes on marketing, politics, or arrangements between software companies, so it’s not always dependent on technological advancement.

Q: Do you see some other competitive alternatives in concurrent programming which may be a game changer?

RG: HTM and STM are already competing with all the algorithms the other lecturers and I have discussed during this Summer School event. They are called lock-free or lock-based algorithms. STM and HTM are already themselves in competition. Whether HTM or STM will win or not, I don’t know, but they’re definitely the new competition. But as of now, there is no other new competitor to HTM and STM.

Q: You said that there is a doubt about whether concurrent programming will be implemented in finance or not. But, if we imagine that it will be implemented in finance: How do you think it will change the financial services or financial landscape? Can you make predictions?

RG: I think the way it could change fields like FinTech is that it’s usually considered a reliable software. Much more reliable because we understand more, it’s simpler to program, the software is more reliable. And as it’s more reliable, then, when you have a lot of money involved, bankers prefer to use this software. This could really have a big impact on financial services because then people can start automatizing more FinTech decisions, if they have more trust in the software. I worked with some banks in Switzerland and for them it’s very important to be able to say: “I trust my software.” And now there are these robots, which make predictions about currencies and everything, and bankers are beginning to trust them more and more. If we use sophisticated technologies that are reliable, then maybe people will use more of these FinTech robots.

Q: You mentioned reliability. There is a huge problem in the financial industry involving testing of concurrent software. Though there are several theoretical approaches, such as verification and some other techniques, in fact, most of them are not adopted by the industry. Do you feel there will be some progress in this area, and what approaches are most promising as far as making software more reliable?

RG: These abstractions like STM, HTM, etc., are important because they make the software more reliable, as the abstractions are very simple, very solid, and easy to understand. It’s already headed in a good direction. The more we use them, the more reliable the software is.

Q: But if STM and HTM will not be adopted as a mainstream, what are the other alternatives?

RG: I think it’s complicated, because today all known algorithms are very complicated. They use locks, use wait-free or lock-free and this is a mess. It’s very hard to prove them and test their correctness.

Q: So you don’t know of any promising results that will increase the simplicity of applying some formal methods, do you?

RG: There is still a gap between what people are doing in research and what people can use to test software in practice. There is still some work to be done to reduce this gap.

Q: On which side this work should be done; the companies or the researchers?

RG: The companies should also invest some energy to understand the verification protocols and to explain to the researches of what is missing, because today the researchers don’t know what the company needs, because the companies are very busy with their work so they don’t take the time to look at what is needed

Q: You mentioned robots. What do you think about it? Everyone thinks robots are going to take people’s jobs.

RG: This is a serious problem, even in Switzerland: I think robots are eventually going to take most jobs. But someone has to be able to program robots to make the right decisions, so I think we still have some future for a few years. *laughing*

The only job that seems to be safe is how to program robots. It seems that the last stage is robots programming themselves. Now they don’t know how to do that yet, so computer scientists still have some time.

Q: Should we keep developing robots?

RG: You cannot stop it. If we don’t develop it here, someone will do it in somewhere else. It’s unstoppable.