IEEE Computer Society Working on Reproducibility of Research

I’ve been worrying about reproducibility of research results for quite a while now (since the late 90’s [2, 3, 5]).  As digital and network technology we developed in the 80s and 90s has been taken up, science and technical research has become digital, computational, and digitally published.  These technical advances are super useful, but they raise many issues for evaluating research results [4].  They also revolutionize the notion of “publication” and “reproducing” research.

So we worry about not just the data and software, but also the computational steps involved .  New technologies may or may not help track the complexities of data and computation underlying specific results (e.g., cloud computing, blockchain).

But all the technology in the world can’t solve the problem.  To make results “reproducible” requires authors to do a bunch of work to maintain and publish adequate descriptions of the technical underpinnings of their claims.  And it requires publishers to publish and archive not just papers, but digital data and metadata.  (I’ll note that universities need to expand their mission to require data and metadata be deposited as part of thesis and other academic projects.)

We’ve been pushing these requirements since the early days of the World Wide Web [2, 3, 5].  So I’m glad to see more and more publishers and professional societies moving to finally deal with these issues.

I should note that the Astronomy community has long led in this field.  For decades now, all major astronomy publications have required that the relevant datasets be deposited in open archives at the time that a paper is published.  (Sensei Ray Plante pioneered some of the early efforts [5].) Well done.

---

The IEEE Computer Society is catching up.

This fall and ad hoc committee of the IEEE Computer Society has published recommendations for bringing their journals and conferences into the digital age [1]. 

The report sketches the interested parties and proposes some steps for the professional organization, which is an important publisher. Ironically, even in this savvy and well-funded professional field, the field that created the digital and network technologies in question, 60% of the publications do not have any processes in place.

In the end, the main proposals are to (1) enable and require submission of data and code along with manuscripts to be published, and (2) to link the archived code and data with the published paper. Just like astronomers have been doing for twenty years.

So I say, “yes, please”.   This was the right idea when we wrote about it last century, and it’s long overdue now.

---

1. Joanna Goodrich, Study Shows Ensuring Reproducibility in Research Is Needed, in IEEE Spectrum – News, September 30, 2021. https://spectrum.ieee.org/study-shows-ensuring-reproducibility-in-research-is-needed
2. Robert E. McGrath, Joe Futrelle, Ray Plante, and Damien Guillaume. Digital Library Technology for Locating and Accessing Scientific Data. In ACM Digital Libraries ’99, 1999, 188-194. http://dx.doi.org/10.1145/313238.313305
3. James D. Myers, Alan R. Chappell, Matthew Elder, Al Geist, and Jens Schwidder, Re-Integrating The Research Record. Computing in Science and Engineering, 5 (3):44-50, May/June 2003. http://ieeexplore.ieee.org/document/1196306/
4. National Academies of Sciences Engineering and Medicine, Reproducibility and Replicability in Science., The National Academies Press, Washington, DC, 2019. https://doi.org/10.17226/25303.
5. Raymond L. Plante, The NCSA Astronomy Digital Image Library: The Challenges of the Scientific Data Library. D-Lib Magazine,   October 1997. http://www.dlib.org/dlib/october97/adil/10plante.html

Speaking of 3D Printed Gizmos

Clearly, the long promised additive manufacturing revolution is at hand today.  Bespoke catalytic converters, and now researchers down the street in Mechanical Engineering at UIUC report on bespoke super performing heat exchangers [2].

The Illinois research creates highly efficient tube in tube heat exchangers, which work by passing fluids over complex a 3D geometry which wicks up the heat. I’m no expert, but I’m sure that the optimal geometry depends on the fluid, the temperatures, the flow, and the materials in the heat exchanger. 

One advantage of additive manufacturing, AKA, 3D printing, is that it can realize complicated geometries that are very difficult to create by other methods.  A second advantage is that the geometry is controlled by software, so it can be programmed and optimized.

Did you say software?  Did you say optimizing?  As a matter of fact, at Illinois we do know how to make clever optimizing software.  Have been doing it for well over fifty years, thank you very much.

The basic idea is to use Genetic Algorithms to optimize simulations of the physics of the fins (which look kind of like snow flakes in cross section).  These fins are combined with a model of the physics to yield a digital model of the theoretical device. The device is then printed out with a powdered metal 3D printer.  Voila!  A bespoke heat exchanger.

This “bespokeness” is one of the neat things about this additive fabrication.  Basically, it’s possible to design and create a piece for each use, tuned to the exact specifications.  This kind of optimizing software isn’t a design for a good heat exchanger, it encapsulates the best design for any heat exchanger you want.

Cool! 

(Well, I guess some parts are cool and some parts are hot—it’s a heat exchanger, after all. : – ))

---

1. Grainger College of Engineering, Illinois researchers demonstrate extreme heat exchanger with additive manufacturing, in Mechanical Science & Engineering – News, September 9, 2021. https://mechse.illinois.edu/news/41505
2. Hyunkyu Moon, Davis J. McGregor, Nenad Miljkovic, and William P. King, Ultra-power-dense heat exchanger development through genetic algorithm design and additive manufacturing. Joule,   2021/09/09/ 2021. https://www.sciencedirect.com/science/article/pii/S2542435121003883

Robot Researcher Reveals All…

…at least, a lot more about his research.

Dr. Ding completed his PhD work on robotics this year just down the street at UIUC Mechanical Engineering department.  He has published results of his work on quadruped robots.

This summer he took the unusual step of releasing some of the “making of” video, recording some of the work as it happened. 

As he says, “publication videos only show success, and the process of advancement (including failures and lessons) is rarely shared among the robotics community. This video, therefore, serves as complementary material showcasing the inglorious yet authentic aspect of research.”

This video is actually quite interesting, and not just for the unintended comedy of the fails.  I actually can understand how it works a lot better from these videos than from the successfully rigged demos of the official videos.

In fact, I really feel sorry for the poor baby Panther-bot when it gets all confused in some of the videos.

Perhaps there should be a channel just for robotics-in-progress videos.

Robot Wednesday

Tip of the Hat to UI Alum, Cuchu Fan

This year’s ACM Dissertation Award goes to Chuchu Fan for her awesome work just down the street at the ECE Department [1]. 

I’ve never met Dr. Fan in person, but I will join with all Illinois alums saluting her great work.  She makes us proud, and makes us all look good.

The work in question is very impressive [2].  Not that I understand the details.  But I understand why it is important and why it is hard.

From what I do understand of the work, it is an interesting case of putting together pieces to make a useful whole.  Basically, she is able to combine formal mathematical descriptions of the parts of a system, including different types of models, with incomplete and missing pieces.  The “data driven” part includes machine learning, naturally, as well as other data sources.

The hard part is to formally prove properties of this complicated and messy thing.

It’s hard.  So hard no one could do it before, at least not in reasonable ways.

But yes, yes she can. And did.

Naturally, this being Illinois, she also built software and actually did it on real life examples. We expected nothing less. : – )

And these methods are widely applicable and far more efficient.  In short, they are revolutionizing the development of autonomous robots and many other devices.

No this is what I call, “a result”! : – )

Well done, Dr. Fan.

---

1. Association for Computing Machinery, University of Illinois at Urbana-Champaign Graduate Receives ACM Doctoral Dissertation Award. 2021. https://awards.acm.org/about/2020-doctoral-dissertation
2. Chuchu Fan, “Formal methods for safe autonomy: Data-driven verification, synthesis, and applications.” Ph.D. Dissertation, Electrical & Computer Engineering, University of Illinois at Urbana-Champaign, 2019. http://hdl.handle.net/2142/106202

University of Illinois Solar Farm Is Online

As planned, the University of Illinois has booted up the second phase of its photovoltaic farm.

This array brings the total solar generation up to more than 25,000 MWh per year, which is something along the lines of 7% of the total electricity use for the campus.  (It’s a big campus.)

If I have my figures right, this is displaces about 1 / 10 the generation from the existing fossil fueled power plant.

The array will be planted with pollinator friendly plants, presumably in cooperation with the Pollinatarium.

A video feed makes clear that this array is, in fact, actively steered to the sun throughout the day.

(Video capture uploaded from here.)

Cool Tunable Ink From Illinois

Ink is one of those things I don’t think about very mush.  It’s just there, in all its pretty colors.

But, obviously, there is some seriously interesting chemistry going on.

This spring researchers down the block at the University of Illinois report on some interesting “tunable” ink [1]. Depending on how it is laid down, it is a different color.

Cool.

The material is called “bottlebrush block copolymer photonic crystals”, which is a great name for a band.  It is laid down by a 3D printer head on a headed surface.

If I understand correctly, the color depends on the speed of the pen and the temperature of the surface.

These factors change the structure of the resulting crystal, which interacts with light to create the apparent color.

I gather that these differences are very fiddly, so it takes some clever work to control the results precisely.  As they comment, this is done through a “high level of hardware/software integration” ([1], p. 2), which is our wheelhouse here at Illinios.

Give me source code and a place to stand, and I will move the Universe.
Suck it, Archimedes.

The current demo is just a start.  The colors are limited, and printing is pretty crude.  But this is a really promising way to go.

Nice work, all!.

---

1. Bijal B. Patel, Dylan J. Walsh, Do Hoon Kim, Justin Kwok, Byeongdu Lee, Damien Guironnet, and Ying Diao, Tunable structural color of bottlebrush block copolymers through direct-write 3D printing from solution. Science Advances, 6 (24):eaaz7202, 2020. http://advances.sciencemag.org/content/6/24/eaaz7202.abstract
2. Lois Yoksoulian, Researchers mimic nature for fast, colorful 3D printing, in University of Illinois Research News, June 10, 2020. https://news.illinois.edu/view/6367/809468

 

PS.  There has to be a great name for a band somewhere in here:
“bottlebrush block copolymer photonic crystals”,

 

VR Archaeology at Illinois

Researchers at one of my old undergraduate departments at the University of Illinois have developed a cool VRArcheology, a “virtual archaeology field school” [1].  Naturally, Sensei Alan Craig is involved.

 

From what I’ve seen, it’s a pretty complete package, including embodied simulations of digging (on your knees), and lab tests.  The VR is part of a class that includes archival research, data recording, and so on, to round out the experience.

OK, the VR doesn’t include the heat, bugs, and sometimes iffy food, and you don’t have to slog off into the boonies to get there.  But it’s probably a good start.

Of course, this is a bit of deja vu for me.  Sensei Craig and I imagined this a long time ago (certainly 10 years ago), and even wrote unsuccessful proposals to do it.

But that was too early for the technology, which is so-o-o-o much easier and better now.  As the press release indicates, this is built out of commonly used game software, which is some of the most amazing stuff out there.  (The last time I look at it, there was a box to check to “create VR version” of your video game.  It’s that simple.)

It is also true that funding agencies and professional reviewers probably had difficulty grokking the technology back then.  But these days, they have seen their kids playing games (heck, they probably play the games themselves), so they see exactly what it can do.

So yeah, this is definitely a great idea.  It was a great idea when I was advocating it way back when, and it’s great to see it come true.

Nice work all.  I’m looking forward to seeing papers and learning how they plan to disseminate this software.

---

1. Diana Yates, Team creates game-based virtual archaeology field school, in University of Illinois News – Research, January 29, 2020. https://news.illinois.edu/view/6367/805645

Tiny Bio Bots From Illiniois

Holy, Moly!  How did they do this?

Researchers down the street at the University of Illinois report on an astonishing “biohybrid” robot [1].

To be clear, there have been plenty of biobots built with muscle tissue, including here at Illinois.

This one is a bit more than that:  they used both muscle tissue and nerve tissue, to create a little bot that moves by muscles under the direction of the nerves.   Whoa!

“This biohybrid swimmer exemplifies a multicellular engineered living system that is developed via a synthesis of top-down engineering and bottom-up self-organization and development.” ([1], p. 19846)

(There isn’t a word in that sentence that I don’t like!)

Fig. 1. Conceptual framework. The embodiment of the envisioned motile bot consists of an engineered scaffold, ECM, muscle tissue, and optogenetic motor neurons, operating in a fluid environment and responding to external light stimuli. Engineered muscle tissue is formed through self-organization of muscle cells and ECM, guided by the shape of the scaffold. Functional neuromuscular units develop in situ whereby motor neurons extend neurites and innervate the muscle tissue. Appropriate design choices can result in a biohybrid machine capable of locomotion actuated by neuromuscular units. (From [1])

This study uses now established techniques for culturing tissues in a 3D scaffolding. In this case, muscle tissue and motor neurons are cultured together and self-organize into muscle tissue, and motor neurons “extend neurites selectively toward the muscle and innervate it, developing functional neuromuscular units.” ([1] , p. 19841)  Whoa!

The research developed computational methods to design an effective swimmer, with a flat head and a flexible tail for swimming.  This involved a lot of cool biomechanical theory and some serious computational modelling. Even cooler, the neurons are responsive to light, so the little swimmer is activated and controlled by light.

They created a real bot, which demonstrated “actual untethered locomotion”—it works!

Well done, all!

---

1. Onur Aydin, Xiaotian Zhang, Sittinon Nuethong, Gelson J. Pagan-Diaz, Rashid Bashir, Mattia Gazzola, and M. Taher A. Saif, Neuromuscular actuation of biohybrid motile bots. Proceedings of the National Academy of Sciences, 116 (40):19841, 2019. http://www.pnas.org/content/116/40/19841.abstract
2. Lois Yoksoulian, Researchers build microscopic biohybrid robots propelled by muscles, nerves, in University of Illinois Research News. 2019. https://news.illinois.edu/view/6367/802738

 

Robot Wednesday

All Hail RHC!

My old mentor and friend Roy Campbell has finally retired from the Computer Science department at University of Illinois Urbana-Champaign.  He taught and mentored hundreds of students, including over 50 PhD’s (including me).  I’m one of many who might not be where I am today without you.

So let me say a belated, Cheers, Roy!

Anyone who worked with Roy knows that he has always had a broad and deep understanding of the future of computing and communications.  Sometimes I wasn’t sure I grokked what he was talking about, but I paid close attention because I usually found out how important it was soon enough.  I also learned that if I thought one thing and Roy thought the other—he was almost certainly right. (How annoying!)

---

1. David Mercer, After 43 Years, Roy Campbell Retires, But Legacy of Innovation, Leadership Will Remain, in Illinois Computer Science – News. 2019. https://cs.illinois.edu/news/after-43-years-roy-campbell-retires-legacy-innovation-leadership-will-remain

 

One Last Black Hole Simulation at NCSA

Another installment of “winding down the National Center for Supercomputing Applications” (other posts here and here).

NCSA was literally created in order to do black hole research, and through the years astrophysical simulations have always been going on.

So it is fitting that the last years of NCSA should support important block hole work.

This month an international team of researchers report a new simulation that has a 45 year old hypothesis about black holes.  (I’m not astrophysicist, so I’m taking the science at a pretty shallow level.)

Bardeen and Petterson predicted that in a spinning black hole, ”the inner parts of a thin disc would align with the [black hole] midplane.” ([1], p.551)

The paper discusses the technical difficulty of simulating this effect, which, I gather, is pretty hairy, requiring high resolution simulation of the black hold and the accretion disk as they move.

The new study was able to achieve higher resolution, including effects of magnetism, which is important [2]

From my own perspective, and the part I understand better, is that this new result was made possible by porting the simulation codes to the Blue Waters supercomputer at NCSA.  In particular, they used the GPU vector coprocessors with a 3 billion point grid.

The paper doesn’t indicate the exact configuration of the computer (c’mon guys, that’s the most interesting part! : – )), but they could have used thousands of processors in parallel.  (Allocations are measured in “node-hours”.)

Now, I know very well that it ain’t that easy to get a simulation like this to run on such a hybrid architecture, let alone run as fast as possible.  So I know they did a lot of fussy work to make sure that everything worked—correctly!—on this massive system.

Altogether, this is a fine capstone to black hole research at NCSA in the last year of operation (as far as I know).

Over the years, NCSA achieved quite a bit, much of it not contemplated in the original proposal (contributions to networking, security, the world wide web, applications from Art to Zoology).  But, as I said, what is was intended to do was advance black hole simulations.  And it did that superbly.

---

1. M. Liska, A. Tchekhovskoy, A. Ingram, and M. van der Klis, Bardeen–Petterson alignment, jets, and magnetic truncation in GRMHD simulations of tilted thin accretion discs. Monthly Notices of the Royal Astronomical Society, 487 (1):550-561, 2019. https://doi.org/10.1093/mnras/stz834
2. Jackson Ryan, Haunting black hole mystery solved with most detailed simulation ever, in CNET – News. 2019. https://www.cnet.com/news/haunting-black-hole-mystery-solved-with-most-detailed-simulation-ever/