Tutorial: the open-source library Palabos in your daily work
Jonas Latt EPFL, Lausanne, Switzerland
Rome, DSFD 2010
www.lbmethod.org/palabos
Palabos: a library for developing LB models, and for running LB simulations
Two development interface
C++: The native interface; powerful and general, but not always easiest to use. Palabos: A scripting interface which combines the power of Palabos with a look-and-feel similar to Matlab.
Distinguishing features
Combines high-performance and interactivity. Offers a convenient platform to develop new models.
www.lbmethod.org/palabos
Example: ow through a porous media
Lets develop a simple application which contains
Creation of the geometry of a porous media. Execution of the simulation. Visualization of the result. Validation of the numerical model.
www.lbmethod.org/palabos
Comment
The Palabos application presented here is written in the Python language and was typed interactively during the DSFD 2010 tutorial. To run it yourself, you need to download the Palabos library at www.lbmethod.org/palabos, and access the le examples/porous.py.
www.lbmethod.org/palabos
The code
#!/usr/bin/python from palabos import * nx, ny, nz = 100, 50, 50 tau = 0.6 lattice = Block3D(nx,ny,nz, D3Q19, BGK(omega=1/tau)) pout.display( lattice[10,10,10].getPopulations() ) lattice[10,10,10].initializeAtEquilibrium(1.1, [0.,0.,0.]) lattice.collideAndStream() for i in range(1,100): lattice.collideAndStream() imShow(lattice[:,:,nz/2].velocityNorm()) lattice[10:30,10:30,10:30].defineDynamics(BounceBack(1.)) dir(lattice) help(lattice.defineDynamics) for i in range(1,100): lattice.collideAndStream() imShow(lattice[:,:,nz/2].velocityNorm()) lattice.defineDynamics(BGK(omega=1./tau)) x,y,z = lattice.meshGrid() media = (x10)**2 + (y10)**2 + (z10)**2 < 10**2 pout.isoSurface(media, [1.]) nobst = 30 r = 8 for cx,cy,cz in zip(rand_int(nobst,nx), rand_int(nobst,ny), rand_int(nobst,nz)): pout.display(Sphere at position, cx, cy, cz) media += (xcx)**2 + (ycy)**2 + (zcz)**2 < r**2 pout.isoSurface(media>0, [1.]) boundary.regularized().definePressureBC(lattice[0,:,:]) lattice[0,:,:].setBoundaryDensity(1.02) boundary.regularized().definePressureBC(lattice[nx1,:,:]) lattice[nx1,:,:].setBoundaryDensity(1.) lattice[media>0].defineDynamics(BounceBack(1.)) for i in range(1,200): lattice.collideAndStream() imShow(lattice[:,:,20].velocityNorm()) pout.display( lattice[media==0][nx/2,:,:].velocityComponent(0).average() ) pout.imagesc( [ lattice.velocity().strainRate().symmetricTensorNorm()[:,:,20], lattice.strainRateFromStress().symmetricTensorNorm()[:,:,20] ] )
www.lbmethod.org/palabos
Step-by-step: Create the lattice
Input
nx, ny, nz = 100, 50, 50 tau = 0.6 lattice = Block3D(nx,ny,nz, D3Q19, BGK(omega=1/tau))
www.lbmethod.org/palabos
Step-by-step: Access a single cell
Input
pout.display( lattice[10,10,10].getPopulations() ) lattice[10,10,10].initializeAtEquilibrium(1.1, [0.,0.,0.])
Output
[ 0.33333333 0.02777778 0.05555556 0.02777778 0.05555556 0.02777778 0.05555556 0.02777778 0.05555556 0.02777778 0.05555556 0.02777778 0.05555556 0.02777778 0.02777778 0.02777778 0.02777778 0.02777778 0.02777778 ]
www.lbmethod.org/palabos
Theory: Collision rules
Each lattice node can have different collision rule
Additionally to providing the algorithm for collision, this operation species information for computing macroscopic variables, rescaling the populations, etc. . . This way of treating algorithms like data is typical for object-oriented programming.
A lattice cell in Palabos looks as follows:
www.lbmethod.org/palabos
Step-by-step: Run and visualize the simulation
Input
for i in range(1,100): lattice.collideAndStream() imShow(lattice[:,:,nz/2].velocityNorm())
Output
www.lbmethod.org/palabos
Step-by-step: Instantiate bounce-back nodes
Input
lattice[10:30,10:30,10:30].defineDynamics(BounceBack(1.)) for i in range(1,100): lattice.collideAndStream() imShow(lattice[:,:,nz/2].velocityNorm())
Output
www.lbmethod.org/palabos
Step-by-step: Create a sherical obstacle
Input
lattice.defineDynamics(BGK(omega=1./tau)) x,y,z = lattice.meshGrid() media = (x-10)**2 + (y-10)**2 + (z-10)**2 < 10**2 pout.isoSurface(media, [1.])
Output
www.lbmethod.org/palabos
Step-by-step: Create 30 sherical obstacles
Input
nobst = 30 r = 8 for cx,cy,cz in zip(rand_int(nobst,nx), rand_int(nobst,ny), rand_int(nobst,nz)): pout.display(Sphere at position, cx, cy, cz) media += (x-cx)**2 + (y-cy)**2 + (z-cz)**2 < r**2 pout.isoSurface(media>0, [1.])
Output
Sphere Sphere Sphere Sphere Sphere Sphere Sphere ... at at at at at at at position position position position position position position 4 1 21 86 24 0 60 40 35 38 32 16 28 26 15 67 42 3 45 7 22
www.lbmethod.org/palabos
Step-by-step: Create 30 sherical obstacles
Input
nobst = 30 r = 8 for cx,cy,cz in zip(rand_int(nobst,nx), rand_int(nobst,ny), rand_int(nobst,nz)): pout.display(Sphere at position, cx, cy, cz) media += (x-cx)**2 + (y-cy)**2 + (z-cz)**2 < r**2 pout.isoSurface(media>0, [1.])
Output
www.lbmethod.org/palabos
Theory: Limited-range interactions
They are used whenever an operation. . .
. . . is non-local, or . . . involves communication between different blocks.
Examples
Non-local boundary conditions. Coupling between components in a multi-component uid. Evaluation of nite-difference schemes, for example for computing velocity gradients.
They are executed either
automatically if they are part of the physical scheme, or manually if they are part of a data evaluation process.
www.lbmethod.org/palabos
Step-by-step: Instantiate boundary conditions for a pressure-driven ow
Input
boundary.regularized().definePressureBC(lattice[0,:,:]) lattice[0,:,:].setBoundaryDensity(1.02) boundary.regularized().definePressureBC(lattice[nx-1,:,:]) lattice[nx-1,:,:].setBoundaryDensity(1.) lattice[media>0].defineDynamics(BounceBack(1.))
www.lbmethod.org/palabos
Step-by-step: Run the simulation
Input
for i in range(1,200): lattice.collideAndStream() imShow(lattice[:,:,20].velocityNorm())
Output
www.lbmethod.org/palabos
Theory: The Palabos collision-streaming cycle lattice.collideAndStream()
The algorithm
1. Iterate once over all lattice cells
1.1 Execute an in-place collision on the cell. 1.2 Execute a streaming towards all neighbor nodes which are in post-collision state (no temporary memory).
2. Execute all automatic limited-range interaction operations
Each operation has its own domain of activity (e.g. the boundary condition applies only to boundaries.)
www.lbmethod.org/palabos
Step-by-step: Compute the ow through a surface
Input
pout.display( lattice[media==0][nx/2,:,:]. velocityComponent(0).average() )
Output
0.00511083245972
www.lbmethod.org/palabos
Theory: Two ways of computing the strain rate tensor in iso-thermal uids
Local approach: get the stress tensor from particle populations
q1 X i=0
ci ci fi
S=
1 2 c2 s
In Palabos, this is dened through the local collision rule
Non-local approach: compute velocity gradients
1 u + (u)T 2
S=
In Palabos, this is dened as a limited-range operation
www.lbmethod.org/palabos
Step-by-step: Compare stress and strain-rate
Input
pout.display( lattice[media==0][nx/2,:,:]. velocityComponent(0).average() ) pout.imagesc( [ lattice.velocity().strainRate(). symmetricTensorNorm()[:,:,20], lattice.strainRateFromStress(). symmetricTensorNorm()[:,:,20] ] )
Output
www.lbmethod.org/palabos
The programmers interface for Palabos users
Developing with Palabos means to implement either collision rules or limited-range interactions. You do not need to be aware of parallelism while developing these ingredients.
www.lbmethod.org/palabos
The Palabos development interface
www.lbmethod.org/palabos
Performance on a small parallel machine: Intel 8quad-core with MPI
400 400 400 domain, D3Q19
Million site updates per second
195 114 60.1 31.5 17.2 9 1 2 4 8 Number of cores 16 32
www.lbmethod.org/palabos
Performance on a large parallel machine: Blue Gene/P 16000 cores
1000 1000 1000 domain, D3Q19
Million site updates per second
8056 4685 2725 1467 790 430 228 256 512 1024 2048 4096 8192 16384 Number of cores
www.lbmethod.org/palabos
The convenient block interface hides a complicated data structure
The internal implementation offers:
Parallelism. Sparse-memory implementations. Grid renement.
www.lbmethod.org/palabos
The idea of sparse-memory implementations:
A given domain. . .
. . . is internally subdivided to remove unused regions
www.lbmethod.org/palabos
A 3D example
www.lbmethod.org/palabos
The idea of grid renement
Example: lets use grid renement with the 2D lid-driven cavity
www.lbmethod.org/palabos
The idea of grid renement
All levels of grid renement are represented on separate layers (blocks of distinct size)
And all layers are put together in a common data structure
www.lbmethod.org/palabos
If you have questions. . .
www.lbmethod.org/palabos
