16/04/2012

Aimofthecourse

Howtodoresearch
GiorgioButtazzo
[email protected]

To provide somehints onwhat to do and what not to do when you areinterested in: g research doing writing papers presenting your results

http://retis.sssup.it/~giorgio/h2d.html
2

Outline
1. Howtosettleforresearch 2. Howtowritescientificpapers 3. TypicalEnglishmistakes 4. Gettingintothepublicationprocess 5. Howtomakepresentations 6. Simulatingasmallconference
3

Meaningofresearch
Research = Re - Search
Re again Search examine carefully, try, test, probe

Research
Detailed and systematic study aimed at discovering new knowledge about our world (scientific approach). Detailed study of a subject aimed at inventing new solutions for specific problems (engineering approach).
4

Scientistsvs.Engineers
Scientists
Interested in the world as it is Goal: understand how it works Discover laws on what already exists

Basicvs.AppliedResearch
Some people like to distinguish between:
Basic research aimed at advancing our knowledge with little concern for any y immediate p practical benefits. Applied research aimed at achieving some practical outcomes that are useful to the society.

Engineers
Interested in changing the world Goal: solve practical issues Invent something that does not exist
5

16/04/2012

Goodvs.BadResearch
Some other people prefer to distinguish between:
Good research: rigorous scientific study aimed at advancing knowledge and producing new results that (sooner or later) will be useful for the society; Bad research: study aimed at producing specific products without long term objectives or general implications. According to this view, a research is applied if there exists someone capable of using the results to build practical things.

Theoryvs.Practice
The believe that a theoretical study is not useful to the society is completely wrong! Whats more relevant for the society? The theory of electro-magnetic field or a practical study to optimize a screwdriver?

PasteursQuadrant
byDonaldStokes,1997

WhatisEngineering?
Engineering is the development of cost-effective solutions to practical problems, through the application of scientific knowledge
Cost-effective Considering design trade-offs on resource usage Solutions Emphasis on building devices, but also methodologies Practical problems problems that matter to people and improve human life Application of scientific knowledge Systematic use of analytical techniques
9 10

High Quest for Fundamental F d l Understanding

Purebasic research
NielsBohr

Useinspired research
LouisPasteur

Low

Bad research

Pureapplied research
ThomasEdison

Low Consideration of use

High

Elementsforasuccesfulresearch
The path to success consists of three simple elements: Find what interests you that you can do well and is needed by the people. - Lui Sha

Elementsforasuccesfulresearch
Understading yourself
your talent your interest

Important questions to answer:

social needs

What is easier for you, writing a complex software program or proving a difficult theorem? What excites you?

social needs
11

What excites the community? Does it fall into your skills?

16/04/2012

Researchstepbystep
0. Finda topic 1. Understand thefieldandthe applicationdomain 3. Formalize li the problem

Researchstepbystep
5. Validateyour solutioninpractice 6. Compareyoursolutionagainstexisting ones 7. Extendthesolutionto moregeneralcases 8. Communicatethesolution totheworld
13 14

2. Think h kof f the problem 4. Findasolution totheproblem

Typicalresearchprocess
start with simple assumptions use examples and simulations

1.Understandingthefield
Why is it important?

idea

model

conjecture

solution

What are the main application areas? Id Identify tif the th key k components t (HW and d SW) of f your system.

more realistic assumptions

generalize

clean up

Whatdisciplines areneededtounderstanddetails? Doyouhavetherequiredexpertise? Ifnot,whichcourses doyouneedtoattend?

1.Understandingthefield
Are there ethical issues involved (e.g., weapons, animals)? Better to clear these things up before starting. T Talk lk to t people l that th t already l d worked k d on the th topic t i to t know tricks, shortcuts, and bottlenecks. Read the documentation related to the main components (HW and SW). You must become the Master on that subject.
17

1.Understandingthefield
Read the literature related to your problem. Search on the right conference proceedings (ask your supervisor about the top conferences in the topic). Conference papers have more recent results. Get familiar with the work done before (if any): use tools, read the source code, run demos, test the system functionality, discover critical points and take notes on strange behaviors.

18

16/04/2012

Commonmistakes
Think, plan, read, and never get into the main game
Reading is important, but it is also important you start experiments, simulations and analysis. You may find issues are quite different than you thought. thought

2.Thinkingofaproblem
What is the novel contribution you would like to give to the field? Make a wish list, from big dreams to smaller (but more realistic) goals. goals Which features would you like to have on your system? Go back to the literature to see if some problem has already been solved.
19 20

When reading papers, be critic, never take everything as a dogma.

Read papers as a reviewer!

Commonmistakes
Choosing a wrong direction
Your supervisor may have the experience to advise on this.

Commonmistakes
The problem has already been solved
Carefully look at the literature!!

Dontgetdesperate,youmay
Findabettersolution;

Too simplistic assumptions

Extendtheexistingsolutionto moregeneralcases; Slightlychangetheproblemto covermorerealisticcases; Findanotherproblem.

3.Formalizingtheproblem
If the problem is too complex, break it into a set of smaller subproblems. Build a model of your system:
Clearly l l identify d f the h assumptions you need d to simplify lf reality (but not too much); Define the metrics for evaluating the outputs of your system and its performance; Define the system interface.
23

3.Formalizingtheproblem
Distinguish variables between:
System parameters (variables you dont want to change); Input variables (primary variables affecting your method) Design variables (variables you want to change to apply your control actions); State variables (variables describing the system state and behavior); Output variables (variables you want to measure to evaluate the performance of your method).
24

16/04/2012

3.Formalizingtheproblem
Find relations among the variables: how outputs are influenced by inputs? If relations cannot be easily derived by the physics, make experiments: p
Decide a single input variable; Decide a single output variable; Fix the others as parameters; Vary the input variable in a given range; Measure the output for each value of the input.
25

Example1
Determine the timing of a traffic lights in a crossroad:
What are the relevant variables? duration of green (G) period ( p (T) ) flow in each road (i) queue lengths (qi) average car speed (v) average delay of a car

S2

STOP
STOP

S1

G S1

S2

26

Example2
Design a communication protocol in a computer network:
What are the relevant variables? number of nodes network topology p gy average load message size message rate network bandwidth end-to-end delay error probability n

Example3
Design a control algorithm for a robotic device:
Relevant variables
Ball & Beam

M T B D p

System parameters ball mass: m ball radius: r beam length: L ball's moment of inertia: J gravitational acceleration: g
27

ball position: ball speed: beam angle: servo angle: sampling period: settling time: overshoot:

x v T

28

4.Findingasolution
Experiments can give good hints for finding a solution. Even theoretical results can come to your mind by first looking g at experimental p data. Sometime, experimental data can be performed to find counterexamples. Counterexamples brings usually bad news, but they are extremely useful to identify critical situations and countermeasures to avoid them.
29

4.Findingasolution
If the solution is an algorithm, evaluate its computational complexity. If it is too high, simplify the assumptions or try heuristic approaches. Write notes and comments every week to monitor i the h progress of f your research: h problems encountered (how you solved them); experiments you carried out; assumptions you are making; scenarios you have considered, etc. Everything will be useful when you write a paper.
30

16/04/2012

5.Validatingthesolution
Implement the solution; Apply the solution to one or more specific cases; Evaluate the performance according to the metrics previously defined, under several conditions (different values of the input or state variables).

5.Validatingthesolution
Often, evaluating the solution on a simulated system is more practical and reasonable. In this case: input or state variables should be generated as random numbers within g given ranges g and realistic distributions. output variables should be averaged on several runs for each input configuration. Remember to record both mean and variance for each set of output values.
31 32

6.Comparingthesolution
Search the literature to find different solutions to the same problem; Identify all the differences (assumptions, model, complexity, performance); Implement the best methods you have found. Run the same set of experiments for each approach. If your solution performs worse than the others for all possible scenarios, then you are in trouble.
33

6.Comparingthesolution
Before giving up, make sure you compare the approaches with respect to different metrics: often its reasonable to sacrifice performance for a lower complexity or a more realistic model. Sometime the improvement is observed only within a certain range of the input variables. This can still be interesting, since that range can occur for particular systems, for which your solution can be more appropriate than the others.
34

7.Extendingthesolution
Go back to the assumptions you made to simplify the system model: Are there assumptions that are too artificial or restrictive? How the solution and the results would change by relaxing those assumptions? How could the model be enriched with more variables describing more details?
35

7.Extendingthesolution
Are there other components to be considered? Is the analysis improvements? tight? Is there space for

Are experimental results complete?

36

16/04/2012

8.Communicatingthesolution
As soon as you have got an interesting result, write a technical report, do not wait until the whole project is finished. Remember to register the technical report and put it on your web b page. If the work is done together with your supervisor, ask him/her to revise it and put it on his/her web page, which has much more visibility to the external world. This ensures you the paternity of the result, in the case somebody should publish a similar work.
37

8.Communicatingthesolution
Then, select a conference with your supervisor and write a paper for that conference. This is important for receiving a quick feedback (normally after 23 months) from a number of reviewers. In the meantime, you can keep working to extend your work. If the paper is accepted, you have to make a presentation at the conference. This is a very constructive experience, which is important for a number of reasons.
38

8.Communicatingthesolution
Why presenting a paper at a conference is important? You have the unique possibility to join a community of experts and exchange interesting ideas; After your presentation, some people can give you a feedback to improve your work; You have the possibility to attract the interest of big gurus in the field and to make contacts with a lot of interesting people.
39

8.Communicatingthesolution
With the feedback you received from the reviewers and from people at the conference, you can think of writing a stronger paper for a journal. paper p will be The issue on how to write a research p covered in detail later on.

40

Generaladvices
If you have a hard deadline, dont rely on someone else. Most students underestimate the importance of physical aspects, such as: g a sensor to acquire q data into y your p program; g ; Using Making an actuator to move and build a function that controls it as desired; Ordering devices and have them on your desk; Making a new printed circuit from scratch; Soldering components and testing them.
41

Generaladvices
Write down an idea when you have it, because you may easily forget it by the time you get to the office. Some ideas come early in the morning, so keep paper and pencil on your bedside table. Some ideas come at the restroom, restroom so be ready. ready Do not be afraid to admit your ignorance. In academic research one is continually venturing into new areas. Perfectionism and attention to detail is a virtue. Taken to excess, it is a vice.
42

16/04/2012

Howtogetideas
Read widely, but remember, reading is not what
research is about. Too much reading could be a major danger that could prevent new ideas to flourish. Make a selective reading: quickly go through the paper to see if there is anything interesting, interesting then look for specific points. Questions to always have in mind:
What is the main problem to be solved? What are the main assumptions.

Howtogetideas
What to read?
conference papers are the most up to date, but often details are available in technical reports. Most of the time, the key idea is not in the details, but in the methodology. When details are important and you cannot find them, contact the authors directly by Email:
Often they feel honored to be considered and will be happy to provide additional information on their work.
44

If you dont understand the math, make a step back to learn and then return to study the approach.
43

Howtogetideas
Reading approach
Some degree of skepticism is appropriate when approaching most papers. Dont trust anything you read. Verify the theoretical results and judge the assumptions trying to imagine them applied to your system.

Howtogetideas?
Thinkofsomepracticalissue
Startfromproblemsthatreallybotheryou: Dissatisfaction isthemajorsourceofinspiration

45

46

Howtogetideas?
Bedrivenbypassions
Passion isoneofthestrongestforcesinNature
Searchthewebusing combinedkeywords,like: music& statistics painting&algorithms biking& sensors soccer& measurement Youwillfindinteresting unexpectedresults.
47

Howtogetideas?
ObserveNature (Animals,plants,phenomena)
AllsolutionsfoundbyNaturearecharacterizedby HighEfficiency Beautyandelegance

48

16/04/2012

Howtogetideas?
Lookintootherfields
Getknowledgefromotherfields: differentproblemsmayhavesimilarsolutions An example:
At the beginning, documents were written using a single font: Courier Many different fonts are available today much nicer to read!

Howtogetideas?
An interesting story
Simply out of curiosity, Steve Jobs decided to attend a calligraphy course, where he learned all about typefaces, fonts, letter spacing, and what is needed to make printing a beautiful form of art. art The course had no practical value that he could imagine, and he took it just for pleasure. Ten years later, his experience was crucial for the design of the Macintosh user interface, which set a standard for all word processors.
49 50

Howtogetideas?
What do we learn? Curiosity means pursuing something that does not necessarily have an obvious purpose. We examine something merely because it seems interesting. Later, we may discover unexpected connections with what we are doing that can form the basis for new ideas.
51

Somehints
Its very useful making lists, and then organizing them. Examples are lists of questions, possible experiments, possible surveys, sources of data, etc. Imagine the results of experiments before making them: if the results were to be ..., then we would conclude .... otherwise our conclusion would be .... Think of an experiment that nobody else has done, and do it. The emphasis here is on doing something because it can be done, not because you actually understand the implications.
52

Whattodoifyoudonthaveideas?
TakeadvantageofbeinginaLab
Talk to people to know what they are doing. Ask successful researchers in your department if they have particular methods for getting ideas. Attend seminars: you can find interest topics and you learn how to present (or not to present). Brainstorm with your colleagues about problems they have. Collaborate with someone on a common problem.
53

Whattodoifyoudonthaveideas?
Criticizeapaperbysomeoneelse:
Are there assumptions that are too artificial or restrictive? How could the model be enriched with more variables, describing more details? Are there other components to be considered? Is the analysis improvements? tight? Is there space for

Are experimental results complete?

16/04/2012

Whattodoifyoudonthaveideas?
Buildontopofexistingwork:
Can I improve the experiments? Can I improve the theory?
From deterministic d i i i to statistical i i l analysis. l i From fixed priorities to dynamic priorities. From uniprocessor to multiprocessor or distributed. From single goal to multiple goals (feasibility + energy)

Whattodoifyoudonthaveideas?
Reducecomplexity
System are becoming more and more complex Sometime too complex to be properly used or fully understood Simplification is a very promising process in the future at many levels:
Less computational complexity Smaller program size Simpler interfaces
56

Can I build a tool that does it automatically?

Seekforsimplicity
Achievingsimplicityisverydifficult,because Simplicityimplies:
deepunderstanding distilling beautyand elegance

Examplesinscience
E=mc2 F=kx V=RI F=ma

Itneedstime,
Goodideasneedincubation anddeepthinking
57

factorial(n) { if (n == 0) return 1; return (n * factorial (n-1)); }

58

Achievingsimplicity
Strategies to handle complexity: Partitioning Isolation I l ti Abstraction

Partitioning
It is the process of dividing a system into independent parts that can be studied in isolation.

59

60

10

16/04/2012

Isolation
It is the process of focusing on a specific component, to understand its behavior in relation with the others:

Abstraction
An Abstraction is a concept that connects a set of items as a group or category. Abstractions reduce the information content, retaining what is relevant for a particular purpose.

ball

61

62

Abstraction
In Computer Science, abstractions allow program designers to separate methods from implementation mechanisms, so that they do not depend on software or hardware: Examples A queue A resource manager A virtual processor

Achievingsimplicity
Techniques to simplify a process: Reduce functionality Hide extra functions Organize your knowledge

63

64

Reduce
Today, new functions are included just because it is possible, not because they are needed. Reduce functionality

Reduce
Crucial issue to be addressed What to remove? Balance between essential and fancy

clock

alarm

date

temp

set

mode

up

down

65

66

11

16/04/2012

Reduce
Simplicity saves you time. And time is value.

Hide
If you cannot reduce any further, hide non essential functions.

clock

alarm

UP

DOWN

Crucial issues to be addressed What to hide? Where to put?

Hide
Extra functions can be revealed only when needed

Hide
Hiding can be hierarchical

69

70

Organize
Group similar items into categories

Organize
Making a taxonomy is crucial for grasping the key features of a set of similar items: State of the art papers Neural learning paradigms Scheduling algorithms Control schemes Poweraware strategies
71 72

12

16/04/2012

Organize
When I started working on Neural Networks:
Back propagation Kohonen networks Self-organizing maps Reinforcement Learning Q-learning

Organize
Slowly some order emerged from the chaos
Learning paradigms

supervised

competitive

reinforcement

Reward-Based Functions Recurrent networks Associative networks Competitive learning Hopfield networks
73 74

Hopfield

Back-Prop.

SOM.

Kohonen.

ASE-ACE

Q-learining.

Organize
The same thing happened on RealTime Systems:
Rate-Monotonic Deadline Monotonic Sporadic Server Priority Inheritance Priority Ceiling Slack Stealer Least Laxity First Fixed Priority Scheduling
75

Organize
The same thing happened on RealTime Systems
Scheduling algorithms
Fixed-priorities Dynamic priorities EDF

Polling Server

periodic

aperiodic

periodic

aperiodic

RM

DM

PS

DS

EDF

LLF

TBS

CBS
76

Organize
Strategies useful for achieving organization: List all the features Sort Group into categories Integrate (reduce to fewer groups)
VERY IMPORTANT

Organize
The key of software organization:

a TAB character is not equivalent to N spaces

13

16/04/2012

Organize
void { int int int sort(int *v, int n) i, j; min, k; temp; for (i=0; i<n-1; i++) { min = v[i]; // find minimum bewteen i and n-1 k = i; for (j=i+1; j<n; j++) if (v[j] < min) { min = v[j]; k = j; } if (k > i) { temp = v[i]; v[i] = v[k]; v[k] = temp; } } }
79

Organize
Something that should NEVER be produced:

// array indexes // value and index of minimum element // temporary variable

// swap v[i] with v[k]

void sort(int *v,int n) { int i,j; // array indexes int min,k; //value and index of minimum element //temporary variable int temp; for( i= 0 ;i <n-1;i++ ){ min = v[i]; //find minimum bewteen i and n-1 k =i; for (j =i+1;j <n;j++) if(v[j]<min) { min=v[j];k=j;} if( k> i){ //swap v[i] with v[k] temp = v[i]; v[i] = v[k]; v[k] = temp; } } }

80

Thebestexample
money

Researchcomponents
researchers students people

teaching Projects

conferences

Applic

papers Spin-off
81

books

tools

Servicesarealsorequired
Reviews
3reviewsperpaperperyear Projectreviews

Researchhierarchy
Full professor Associate professor Find industrial contacts Meet with industries Propose projects Get money, get positions Teach courses

Teaching
seminars,lectures,courses Takeitasapleasurefortranferingknowledge
Assistant professors

W it Papers Write P Write Proposals Organize demos Teach courses PhD Undergrad students

Post Doc

Organization
conferences,worshops,specialtracks,demos, summerschools
83

14