Monday, July 07, 2014

Things I didn't know before becoming a professor: #4 Different types of collaboration

There's a romantic and resilient fantasy of the scientist toiling away in isolation.  He (sad to say, it's almost always he) emerges from the lab, bleary-eyed and triumphant and shares his genius with the world.

Da Vinci, Frankenstein, Edison, Einstein, Every Evil Genius, Zuckerberg

Our science narrative has expanded to pairs of geeks in garages

Hewlett & Packard, Gates & Allen, Jobs & Wozniak, Brin & Page

but hasn't yet embraced the fundamental reality of modern science: most work is done in collaboration.  Most great big ideas are the product of many great small ideas.

This is why there's no satisfying answer to questions like "who invented the atomic bomb".  Was it Fermi? Einstein? Oppenheimer?   Of course Al Gore didn't invent The Internet, but who did?

As a graduate student, you are, at worst, Igor, the hunchbacked assistant doing all the work and getting none of the credit.  At best, you are a near-equal collaborator with your advisor.    But there's a sea of collaboration waiting once you're out of the nest and on to the next step.

Since becoming a professor I've noticed a handful of types of collaboration. From intimate to remote, they each have a role to play.

Peer Collaboration
This can be the most satisfying and most productive way to work with a partner or team.  You sit in the same room, or with a chat window open.  You talk through ideas, big and small. You tackle the parts of the work you're most suited to, or most excited by.  You debug code together. You run experiments together. Try to make sense of your shared results. You iterate over writing papers together.

It can also be the most frustrating.  A group only walks as fast as its slowest member.  If you don't work well with your partners, for any reason, the stress is visible almost immediately.  If the partnership is strong, you figure it out, and get back to work.  If it's not, you'll know pretty quickly.

Mentor-Student Collaboration
There's a similar intimacy in this collaboration, but there are some very clear differences.  First the pace of collaboration is typically slower.  At slowest, you may only work with your student only once a week.  Even if you're in regular email contact it's only a couple of times a day.  Typically it's only during the most intense periods of an evaluation or paper deadline does the collaboration reach the level of immediacy and proximity of peer collaboration.

The division of labor here is more driven by the hierarchical nature of the relationship.  The mentor typically has a broader vision of the work, while the student has a better grasp of the details.  (Or if they don't, it's part of the responsibility and training to learn about the details.)  The mentor has the responsibility of contextualizing the work, either into a broader research agenda that will include the students thesis as an offshoot, or into a broader project or publication.   At the start of the relationship, the mentor is typically responsible for guiding the research, determining which research questions are most fruitful and

Over the course of graduate study, a good mentor-student collaboration will take on more characteristics of a peer collaboration, with the student ultimately initiating most of the research ideas.  But there's always going to be a level of deference that the mentor receives.

-----
As a graduate student, I had a lot of experience with the previous two types of collaboration. I even had some exposure to the next three, but so much as to be able to identify the differences, or how they impact the work.
-----

Project Collaboration
On medium-to-large multifaceted projects there are frequently multiple PIs (principal investigators) each with independent research groups.  Collaboration across sites towards a common goal is different and challenging.  The PIs coordinate with some frequency.  There are bureaucratic reasons for this -- mostly around budgets and report writing -- but coordination between students is much less common.    Each of the individual research groups have their own broad responsibilities, timeframes and agendas.  Each are balancing a number of projects, student milestones (qualifying exams, etc.), and specific personnel decisions.  

The biggest difference in this kind of collaboration, in my experience, is the pace.  The interaction between groups when it comes to research is infrequent.  Sometimes just a few times a month.  Another difference is in the level of detail.  When the groups share their work, it is usually at a relatively high level -- far removed from unsuccessful approaches and interesting failures.    There are opportunities to get advice at a high level about directions and plans, but close collaboration about specific decisions and approaches is much less common.  

This is completely understandable. The nature of this collaboration demands it.  These projects are big enough to demand a large number of people working on them.  This task oriented partitioning into research groups is probably the best way to manage the effort.  

However, there's the time when the rubber meets the road.  This is usually in the form of deliverables, where you have to send a report or code to a funder, evaluations, where all of the moving parts that the groups have developed have to work together, or presentations at site visits or PI meetings, where you have to show that you're all working towards a common goal.

Often, these moments demand closer coordination and collaboration.    Having made a few mistakes. The lesson I've learned is to stop and drill down.  Take meetings offline to deal with discrete questions and issues.  Get the students involved in these meetings.  If at all possible, visit your partners. With your students.  One hour face to face can take the place of a few days worth of emails.

Social Collaboration
Still another step more removed is the kind of collaboration you have with the scientific community which you're a part of.  This is the sort of collaboration where you have a conversation with someone about an idea that one of you is chewing on.  You bat the idea around for a little while, maybe hear about a technique you didn't know about or hadn't thought of, and then the conversation moves on.  I've found that this kind of collaboration occurs with someone you know.  Either you know each other's work, or you have a mutual colleague to bring you together.

Most of the time this type one-off micro-collaboration ends with a single conversation.  Sometimes it results in shared data or tools.  But given the right alignment of circumstances it can foster a more serious coming together. Usually just an extended conversation over email, or a phone/skype conversation, but if fate is really on your side, this can grow into a paper or a grant proposal. But by this point, you've switched categories and you're working much more closely.

I find that this collaboration is most available during visits to other institutions and at conferences. Frankly, this kind of collaboration is often the most valuable thing I take away from conferences.

"Public" Collaboration
If you're doing it right, people are going to know about your work.  People will email you with questions and comments.  You will get queries about how to use a tool you wrote, or (maybe more frequently) a bug report or feature request.   Through this communication you are supporting other people's research.  This rarely turns into a more substantial collaboration.   But if it means that people will cite your work.

Here, the collaboration is somewhat lopsided.  Essentially your prior effort of writing a paper or tool are supporting someone's current research.  But this needs some additional support from your current self, either in clarification or amendment.   The upside is that if the system is working, you can garner this support as much as you give it out.

Here, I find the challenge is to appropriately prioritize these interactions.  More frequently than I'd like to admit, these are the emails that filter to the bottom of my inbox.

If you're reading this, and I still haven't gotten back to you, I'm sorry.  I know these interactions are important.  I'm working on it.

Wednesday, May 21, 2014

Things I didn't know before becoming a professor: #3 How to run a research group

At this point, I've figured out my classes and secured a little bit of funding.  Now it's time to staff up the group and get to the work of research.

Every researcher I know keeps a list of open questions and tasks that they want to get to next.  This list is a best friend, and a worst enemy.  This list is a hydra.  Every time an element is taken off, two more appear in its place.  To tame the beast, it takes a group effort.

It's no revelation to recognize that management is different from labor.  I'm sure if I had an MBA instead of (in addition to?!) a PhD, some of these issues would be more intuitive.  But I didn't, and most professors don't.  We're self-taught.

When the research group is firing on all cylinders, it feels like my work is having a multiplicative effect, that I'm lifting more than I possibly could alone.  When it's not, it's demoralizing. I try not to get sucked into this, but there are days when it feels like I could do more in a cave with a laptop, an internet connection and coffee.

These are some of the things about running a group that I didn't know before becoming a professor.

How to organize a research agenda

At my thesis proposal, Kathy McKeown asked me the hardest question: "What do you want to be famous for?"   It totally blindsided me.  I laughed uncomfortably, and looked to my other committee members for acknowledgement that this was an outrageous proposition.  No help was coming.  I have no idea what my answer was, but it's the only question I remember from my candidacy exam, proposal or defense.

A clear answer for this question, an organizing principle for the research that your lab will do, brings clarity and identity to the group.  It represents a research agenda that guides the work.  A clear research agenda serves a similar role as a mission statement and business plan.  It communicates your goals internally and externally.

This also helps with recruitment -- if students and postdocs know exactly what kinds of questions you are interested in, you will attract people who are interested in these questions, and people who are capable of working on them.  The same principle applies to students who are looking to do independent studies, undergraduate or master's theses with you.

I've generally under-appreciated the value of this focus.  Personally, I've benefitted from broad research interests.  I get a lot of satisfaction by working on a lot of different things. My most cited paper wasn't even cited in my dissertation. But I think I've also suffered from this generalist approach.  I may have had greater impact on the questions I find most interesting had I limited my activities on the second tier.  I may have made more progress in getting "famous" for something.

How to engender community and collaboration

Being a graduate student can be isolating.  There are some classes.  A weekly meeting or two that you need to show up for.  But most of your work is alone.

How do you take a group of students who are organized only by you, the professor, and help them communicate, support each other, and become colleagues?  I certainly had no idea.

Some things I've tried:

Taking everyone to lunch.  this didn't work so well.  it probably would work now, but at the time the students didn't know each other well enough and it was just pretty awkward.

Shared tasks.  Inviting a group of students to work together on a short term project is the single best thing I've done to get students to trust each other and exercise their basic research skills.  I use the interspeech paralinguistics challenges for these.  I also invite other computational linguistics students to participate from the linguistics and cs programs.  It's a really fun exercise and the students get a lot out of it.

Weekly lab meetings.  One student gives a talk every week. It gives a safe space for students to practice giving presentations (and giving feedback on presentations), and it makes sure that everyone knows what everyone else is working on.

How to make and manage a budget

This is my least favorite part of the job. Making a budget is fine.  It's just making puzzle pieces fit.  But inevitably there is some wrinkle to sticking to it.  Travel costs are more or less than budgeted.  At CUNY tuition is highly variable based on what year a student is and how many courses they are taking.

And then there is interacting with admins, grants, financial officers and contract lawyers.  These people are invaluable.  When the relationship is good, your life is easy.  When there are delays it can make sticking to the timing of a budget very very very difficult.

Having an eye for detail and the patience to use it really comes into play here.

Not everyone knows what I know

The next three observations are less about specific skills, and more about the personal effects of leading a group.

As a new professor with new graduate students, you know more than any other research group member.  This isn't a "smartest person in the room" thing (though sometimes it is), but an experience thing.  The professor has read more papers, seen more talks, (may have executed more experiments) and knows more about the topic.

The list of things that you want to work on next are based on your previous work, your thinking and your instincts.

In order for a student to pick up and work on a task, they need to get up to speed, and fast.

It is too easy to forget that the people working with me don't know what I know. 

How to delegate

It became clear to me early on that not all CS professors still program.  This struck me as a scary idea.  First of all, I like to program, and I'm reasonably good at it.  Secondly, it seems like distance between a professor and the nuts and bolts of research is a dangerous thing.

I was talking with Dan Ellis shortly after I had realized this.  (Nevermind that Dan is a EE professor.)  I asked him if he still coded himself.  He had just finished a project over the summer and was happy with it and the process, but then he described it as "indulgent".  why "indulgent"?  because, according to Dan, he should be working on writing grants, papers and reports, executing the broad vision of the lab, while the students benefit from the programming experience. (Dan, if you read this, please forgive mistakes by paraphrase, and the fog of memory.)

The responsibility to delegate work to students is one that I haven't quite internalized.  I still run a lot of experiments myself. I write a good deal of code still.  And when we're under close deadlines, I have a tendency to tell students to walk away, and handle the final push myself.  I'm not yet prepared to call this "indulgent", but it I'm ready to acknowledge it as a bad habit.

How to be a mentor

Internalizing the role of "mentor" is much more difficult than "teacher".  Teaching is something that you do a few times a week, but it's a hat you take off.  Mentoring, or advising graduate students, is an ongoing process.

I'm the most visible example of what a professor is for my students.  How I do this job impacts how they will.  (Just as my advisor is my first and best example that I draw from when advising students.)

My behavior as a mentor impacts how they'll behave as students. If I respond to emails at 1am, I will get emails at 1am.    If I don't communicate what I expect from them, they won't know.

In addition to my advisor, I've had a number of important mentors.  There are two lessons I've taken from those relationships.

Protect your advisees.  There are plenty of external challenges -- defenses, negative paper reviews, job postings, evaluations -- that a student will participate in.  They do most of the work.  I feel like it's my job to make sure they're set up for success.  (Only submit papers that you would accept. Don't volunteer them for too many responsibilities.)  When things blow up, absorb the blow.  Outwardly, take responsibility for mistakes, figure it out with the student internally.  If the students feel safe, they'll be enabled to do good work.

Be a happy warrior.  There are parts of the job that are difficult.   It takes long hours and drive.    And you're in this job for a reason.  You want to discover, and share those discoveries. You want to contribute to Knowledge.   As a student, I fed on the enthusiasm of my mentors.  While it's easy and appropriate to feel a kinship with students, I feel a responsibility to set a tone that this is valuable, but also this is fun.  And it is.


Monday, May 05, 2014

Things I didn't know before becoming a professor: #2 How to Write a Grant

To get tenure in computer science, you have to secure external funding.

I don't know how important this is in other disciplines.  I'm sure there are administrations that will say that external funding is only one of a number of criteria that determine tenure decisions.  I'm sure they're right.   But without funding, you're at a severe disadvantage. You can't work with grad students (as easily -- some programs provide student funding through other means).  Travel to conferences -- where you present your work and solidify relationships with important people in the field (who may end up writing letters in support of your promotion) -- is expensive.  And then there's the brass tacks.  Depending on institution and some uninteresting details, the administration receives somewhere in the ballpark of 1/3 to 1/2 of the funds that you secure.  

Before becoming a professor, I hadn't written a grant proposal. I had read a couple of successful proposals that I had worked under as a student, but I hadn't been a part of the preparation of any.

A little bit of context before I share what little I've learned about writing grants.
I've been pretty successful in securing funding over the last five years.
I don't expect this trend to continue forever.
I recently received an NSF CAREER award.
But my previous two proposals were not funded.
I'm hardly an expert, but I've gotten better since starting.

Here are a few things I didn't know

How many funding opportunities there are? 
I knew about NSF and NIH. I knew about DARPA because I worked on a grant as a student.  I didn't know how many programs NSF has (there's no way I know about all of them).  I didn't know about the other DoD agencies that fund basic research.  I still don't know about all of the foundations and industrial grants and awards that are available.   I'm sure I'm still missing some, but each of IBM, Google, Microsoft offer programs to support faculty.  On top of that, there are institutional awards -- CUNY has a handful of them, and if we do, I'm sure everyone else does.  This doesn't mean that it's easy to find funding.  But by keeping an open eye, asking around, and being lucky enough to be asked to join grant writing efforts, I've been surprised by how many opportunities are available (in this field).

How to tell the difference between a good proposal and a not-good-enough proposal
Friends and colleagues will offer to share successful grants with you.  Take them up on it.  But learning from only positive examples is challenging.  It can be challenging to figure out how to translate structure and presentation cues from one grant to another.  Also, it's not always clear what made these examples impress the reviewers.   If you're so bold, ask these people to share the reviews along with the grant.  I haven't tried this, and I'd have to trust a person quite a bit to share my highlighted flaws with them, but give it a shot.

Still better is getting to look at unsuccessful proposals and their associated reviews.  Everyone you know has one or two of these kicking around.  Funding rates are pretty low, say 20% or so, so most people will have a stack of failures for each success they have.  But again, it takes moxie to ask someone for this, and a lot of trust to offer it.  (I don't know if this happens ever, but I'd love to hear if other junior faculty were able to read unfunded proposals as they started writing themselves.)

The best way to see a lot of examples of grants, both positive and negative, and their reviews is to get on a grant reviewing panel.  NSF Program Managers like to include junior faculty in their panels.  Call one who funds work like yours, and have a conversation.  I was invited to one in Spring 2010.  It was an opportunity to closely read about 5-6 proposals, and more casually review another 12 or so.  Seeing all the ways a basically good idea could not get funded was eye-opening.  Generally, the quality is high, so the differentiation between successful and unsuccessful proposals can be the depth of their weaknesses rather than the height of their strengths.

That writing a grant is writing science fiction
It's believable, near-future science fiction, but still in the genre.   You're describing something that doesn't exist, but will in the next few years.

This perspective has made the process of grant a lot more entertaining.

Science writing has a narrative arc.  The introduction and motivation of your work should be exciting.  The feeling I want to leave a reader with is somewhere between, "Of course that's what will happen", and "Wouldn't it be cool if that's what the world was like."  Balancing these two extremes is the difference between the Scylla and Charybdis of incrementality ("boring") and overreaching ("i don't believe you can do this").

How to keep it together after getting a grant rejected
It's only natural to be offended, embarrassed, insulted, frustrated, and sad when a proposal that you spent months of your life on is not funded.   The panel was clearly full of idiots who didn't understand your genius.  There was some mistake, if only they had read more closely they would understand how important this work is.

Here's what rejection has taught me:

Take a deep breath.

Most proposals aren't funded.

The folks who review your grants are almost universally qualified to do so.

If a proposal is funded, you are not a genius.  If a proposal is not funded, you are not an idiot.

The sooner you get over it, the better. Remember, it's the work that's being reviewed, those crammed 15 pages of science fiction, not your identity.

And there's a silver lining: almost always they will include a stack of written reviews. These reviews are a gold mine.  Treat them as a genuinely helpful consolation prize.  If you don't understand what they're saying, you can probably talk to a program manager about them.

When revising a proposal, I take all of the reviews, and cut out everything positive that they had to say.  It's too easy to be comforted by that.  What I really need to know is what didn't work.  That's what needs work.

How to find writing advice
I still don't have a lot of confidence in my grant writing.  But there are a lot of people who do, who will share their advice with you.  Almost all institutions have grant writing workshops.  NSF hosts workshops.  There are posts like this, and this, and this, written by people who are truly qualified about the process to really steer you right.  Writing with a group of people helps.  Collaborative proposals come with their own logistical challenges, but it can be helpful for getting feedback on the writing.

Some graduate students write applications for fellowships, internships and other sources of funding.  This is great practice, but this is different from the grant process.  Maybe the best way for students to get exposure and experience at grant writing is through their advisor's proposals.  I think some professors work with their students on their proposals.  If the student is a strong enough writer, and far along in their dissertation, I could see this making a lot of sense for both the professor and student.

Sunday, April 27, 2014

Things I didn't know before becoming a professor: #1 How to Teach

In order to be a professor* you need a PhD.  The two responsibilities you have as a professor are to do good research and to teach well.

To get a PhD, you need to do good research, so you're well equipped to handle this.

To get my PhD, I did not have to teach.  I had TA'd a few times.  But I had put in 10 years in college/grad school.  I had taken countless classes, some with great teachers, some less great.  So I figured no big deal; I can teach.

I've seen good movies.  I've got a good story.  I can make a good movie.

These are some things I didn't know about teaching before becoming a professor.

You are television.
I learned this from Michael Cirino in a very different context.

For the time that you are in front of students, you are television.  You are putting on a show. If your students don't engage with you, your students won't learn anything beyond what they get in a book.     That's not to say that you're entertainment, but you are performing.

My act isn't strong yet, but it's getting better.

Don't skimp on the basics.
In an early version of a Machine Learning class, I decided that I wanted a lecture on spectral clustering. It was a topic I was interested in, but didn't have a ton of experience with. I read a lot. I worked out math.  By the time I had a lecture ready, I was feeling good about it.  I went to class, delivered my A material....blank stares.  It was a total dud.

It wasn't that the lecture was bad, but I hadn't earned it.  Because I wanted to get to a topic that was exciting to me, I had rushed through or omitted a lot of background material.  I had completely set myself up for failure.  When I went back to figure out where I had went wrong, I realized it was months earlier, when I was planning the syllabus.

Every time I teach a new class, I tell myself that I'm going to have half a dozen or so lectures ready to go by the first day of class.  Sometimes I hit this number, usually I don't.  But I've come to realize that lectures are a lot easier if you have a clear structure ready for the class.  Lately, I've become less concerned about having complete lectures prepared.  Instead of writing a few complete lectures, I try to have a fairly detailed outline of every class.  This helps me focus on the big picture.

When I have a clear direction for how the pieces fit together, the course is better.  Even if that means I don't get to the most exciting material, or have an excuse to learn something new. 

Preparing a lecture takes an unbelievable amount of time.
Truly unbelievable.  In my first semester, it took 8-10 hours to prepare each 75 minute lecture.  Add in writing assignments and exams, grading, office hours, and 2.5 hours in front of students.  Thankfully, I was only teaching one class.  But at CUNY a full load is 3 classes in a semester.  

Most of your time is spent with students who are struggling.
"Do you learn from your students?" No. I teach them.  "Are you inspired by your students?"

My PhD students are great.  They bring some exciting ideas and papers, and there's a collaborative learning that happens there.  They inspire me and drive me.  Absolutely.

Students in class, I have a very limited and lopsided relationship with.  I don't think I've ever had an office hours meeting with a student that took material from class and took it a step further.  It's almost always something to the effect of going over material from the previous lecture or exercise in more detail.  There's immense satisfaction from guiding someone to understanding challenging material, but it takes a lot of time.

Students cheat.  A lot.
I have had a student cheat in almost every course I've taught.  (This isn't unique to CUNY. Ask around.)

The cheating meeting is the most emotional human experience, I've had with anyone other than a family member or romantic partner.

The student usually cries.

The student usually tries to negotiate out of the repercussions.

Denial?  Not so much.  Most people own up to it pretty quickly.  Forceful denial is a red flag for me. Be open to the possibility that you made a mistake. Maybe one party knew about the cheating and the other didn't.  Maybe the similarities between two assignments really were random.

I've had over a dozen of these conversations.  Here's my best advice:  Be prepared. Be able to clearly explain how you know cheating happened.  Be able to point to your syllabus and university student handbook about the penalties for cheating. Absolutely document everything you can about the exchange.  Send the student an email after the fact, recapping the major points.   Expect that the student will scramble for an out -- some way to lessen the impact -- don't let them.  At this point, I have a loose script. It's almost as formulaic as a five-paragraph essay.

A. It's clear to me that you cheated on this assignment/exam.

B. Here's how I know.  (it's about here where they usually admit to it.)

C. Because of this you will be getting a zero on the assignment/failing the class/getting expelled, and I will be send a letter with this information to the department.  (Or whatever your policy is.)

Put it in writing.  Get it in writing.
Almost all I knew about teaching I learned from a video (VHS!) I was shown while at Columbia.  This was an impromptu sharing, it seemed like it was a tape that was passed around the CS department and shown by professors to their grad students.  It was a lecture by John Kender called something like "How to Teach".  It was fantastic.  It's similar to this video on iTunes.  Before you teach, watch it.

The most significant lesson I remember from this lecture was that a syllabus is a contract, an assignment is a contract, an exam is a contract.  It is your responsibility to outline the terms of this contract as clearly as you can.  This is what you will learn from this class. This is what to expect from this course, assignment, exam.  If you do this, you will get this grade.

Most of the difficulty I have had with students and disputes can be traced back to not being rock solid in the language used in a syllabus, or on an assignment, or (and this was a surprise) not putting things that were discussed in a meeting, in writing.

If you make an arrangement outside your syllabus with a student around any element of your course, shoot them an email after the meeting recapping what was discussed.  I didn't know this before teaching, but wish I had.

What now?
I've definitely become a better teacher over the last five years. I've made a lot of mistakes and I still do.

Each time I repeat a course, I toy with its structure. My boiler plate syllabus has gotten tighter.

The broader point is when I started, I was starting cold.

There must be ways, programs, seminars, etc. that aim to teach people how to teach.  I was barely exposed to any as a graduate student; I know many of my peers weren't either.  Very little was available before I was in front of students for the first time.

It's easy to gripe about a system that left me unprepared, but now i'm on the other side of the equation. I have graduate students, some of whom will go on to be professors.  What can I (and my institution) do to make sure they have the skills to be good teachers when they land tenure-track jobs (which of course they all will).

Practice. Practice. Practice.
For me, becoming a better teacher has taken practice.  I think that's the only way to learn how you are going to teach.  And there aren't enough opportunities to practice this.  (I've taught Algorithms 4 times, and Machine Learning 3.  That may sound like a lot, but it's only 2 or 3 opportunities to revise structure, lectures and graded material.)

At CUNY, graduate students teach a lot**, so many of my students will have experience in front of students.  The downside to this is 1) teaching takes a lot of time.  This means that they're not focusing on their research.  and 2) Often they're not given the responsibility/opportunity to design the class themselves.  Instead they teach a section of a larger course with a fixed set of assignments and exams.  This leaves students with plenty of experience lecturing, and leading discussions, but less experience with the mechanics of running a course (which is where your teaching lives or dies).

I think one solution might be to have graduate students prepare and teach mini-courses, complete with syllabus, and graded assignments.  These should be short, maybe 6 or fewer meetings over a month or so.  This keeps the workload more manageable compared to teaching a full course.  But it would allow students to practice structuring material, and writing homeworks and exams.  They shouldn't be offered during regular course periods, but in summer or between terms.

I think the best approach would be for this mini-course to be on the student's dissertation topic.  First of all, they'll already know a ton about it.  Second, if they go on to a tenure-track job, chances are they'll have an opportunity to reuse some of these lectures, either in a(nother) course of their own or at a conference tutorial.  Third, lecturing on a topic, and fielding questions can bring to light all the things you don't know or are unsure of.  But the practice would be useful even if it was on some other topic.

The biggest problem I see with this idea is getting the incentives right.  To teach something like this takes a lot of work, and there's little reward.  Moreover, there's little incentive for other students to take one of these mini-courses (and to do the homeworks/assignments).   MIT has a thriving IAP program with a ton of activities and minicourses ranging from the technical (some for credit) to the slightly absurd to one of my favorite things.  The IAP is well established in the MIT culture.  Can something similar be started up elsewhere?

There's no way to do this through the university registrar without a lot of bureaucracy.  However, if a department, or division, were to unofficially "bless" this kind of activity by 1) including a list of course offerings, 2) document who taught what when, and 3) conferring completion "certificates" (and 4) finding teaching space), the publicity of a program like this could encourage students to participate on both sides.

There are a lot of reasons that a program like this would fail to get off the ground, but if there were a mechanism for graduate students to get practice running courses in a relatively low-risk environment, I am confident that they would be better prepared for tenure-track positions.

I would have been.


* tenure-track
** maybe too much, but that's a different discussion

Thursday, April 24, 2014

Things I didn't know before becoming a professor (and that i'm still not very good at)


July 2009. I deposited my dissertation.

September 2009. I started a tenure-track position at CUNY.

Coming up on the close of my fifth year, I'm convinced of a simple proposition.

I was unprepared to be a professor.   

This is not a reflection on the academic preparation I had, or that my weaknesses went unnoticed by the search committee that hired me.  Neither do I think I've done a particularly bad job over the last five years.  Rather, there is a disconnect between the skills that are required to become a professor and the skills that are needed to be a good professor.

To land a tenure-track position you must:
  1. get a PhD
  2. get a solid publication record
  3. get good recommendations from important people
  4. give a good job talk
  5. be personable enough to not ruin your visit to campus
  6. get lucky (there are fewer tenure-track positions than in the past)
(If you're very lucky, you've got some funding when you walk in the door.  But this is a catch-22.  It's really hard to get funding until you're already a professor.)  

The only one of these that is non-negotiable is having a PhD.  
In order to get a PhD* you must:
  1. do good research.
  2. survive on little money and less sleep
That's it. 

Over the last five years, I've repeatedly found myself in situations where I have no idea how to do things that I am expected to do well.  I was a good candidate for a tenure-track position, but a mediocre professor. 

Here's an incomplete list of things I didn't know before becoming a professor (and that I'm still not very good** at).
  1. how to teach
  2. how to write a (successful) grant
  3. how to head up a research group
  4. that project collaboration is different from research collaboration
  5. how to manage my time
In all new jobs, there are things that you have to learn how to do, skills that get developed through practice.  But this isn't figuring out where the closest printer is, or how to fill out a TPS Report.  Most of these skills are central to the job.  There is a disconnect between the requirements to get a tenure-track job and the skills needed to do it well.  

Over the next few weeks, I'll drill down on each of these.  Hopefully, this will be some comfort to other pre-tenure faculty members, and a preview for graduate students.  It's helpful to acknowledge these challenges and the gap between what we expect from graduate students and professors.  In a perfect world, this points to opportunity for graduate programs (including mine) to provide more support to better prepare good graduate students to become good professors.  

* specific requirements vary by institution
** I've gotten better... But mostly through missteps and course corrections.

Wednesday, June 05, 2013

Deep Thoughts on ICASSP 2013

ICASSP 2013 is wrapping up today in Vancouver.  Unfortunately, I missed the last day (and sessions on speech synthesis and prosody that I would have enjoyed).  But a wedding on Saturday brought me back a day early.

I hadn't been to ICASSP before, mostly due to timing oddities and writing grants over summers rather than writing papers that would hit the deadline.  It is a very large conference.  About twice as large as Interspeech.  But the scope is also much broader.  Speech and Language work made up at most 30% of the work at the conference.  And even this is generous, including machine learning, and other work on audio.

So take this recap with a grain of salt.  I missed the last day of the conference, and my impressions are speech focused.  (I think I've described all conference recaps as blind-men-and-the-elephant problems and this one is no exception.)

Deep Learning.
OK, I pointed out that Deep Neural Nets were a "hot topic" at last years Interspeech.  It's hard to believe it's possible, but they're even hotter now.  Geoffrey Hinton gave the first plenary talk.  This was followed by an oral session called "Automatic Speech Recognition using Neural Networks", which was followed by a Special Session titled "New Types of Deep Neural Network Learning for Speech Recognition and Related Applications".  The next morning, you could attend "Acoustic Modeling with Neural Networks".  And this is just at the session level.  Even more applications of multilayer neural networks were scattered around other oral and poster sessions.  Some of these oral sessions were so crowded that people were standing along the walls and sitting in the aisles.  Nothing else that I saw received nearly so much attention.

It's easy to view "deep" learning as a silver bullet -- the next great machine learning that will solve all of our problems.  It's almost certainly not.  However, a wide array of research groups are seeing similar impressive performance gains by using deep network models for a broad spectrum of spoken language processing tasks.  This is especially true for acoustic modeling in speech recognition.  Given this, deep learning shouldn't be ignored.

Hinton's coursera course is a solid place to start. (Though resist drinking the kool-aid.  To my mind, perceptrons are bad approximations of neurons and worse approximations of the brain, and do little to advance our understanding of human intelligence.)

Another highlight
One paper which caught my attention for its simplicity came out of Google: "Language Model Verbalization for Automatic Speech Recognition".  Essentially "verbalization" is defined as a sort of inverse text-normalization. In text normalization for speech synthesis we have to translate "10" to "TEN", and "7:11" to "SEVEN ELEVEN" or "ELEVEN PAST SEVEN".  For ASR, the idea of verbalization is to convert decoding output of "SEVEN ELEVEN" into "7:11" or "7-11".  Why bother?  Well, Google (and everyone else) has big language models based on text data. You could run a text normalizer over all of this data, but the proposition here is to convert the ASR output into a form that looks more like the source material in your language model.

The Verbalizer solution to this problem is remarkably elegant.  A traditional WFST decoder can be expressed as D = C • L • G, where C comes off the acoustic model mapping context dependent to independent phones, L is the pronunciation model and G the language model.  The "Verbalized" WFST model includes a WFST V which maps ASR realizations like "SEVEN ELEVEN" to text-like realizations like "7-11" or "7:11" (and since it's a WFST it can do both simultaneously).  The new decoder looks like D = C • L • V • G.  No fuss, no muss.  Except that you have to write Verbalizer rules by hand.

The paper focused on terms involving numbers, but the framework is very extensible.  And it's great to see work coming out of Google that doesn't have Google-scale data as a prerequisite.

Meta-comment
The acceptance rate at this years ICASSP was 52%.  This means that the ICASSP and Interspeech acceptance rates are identical for the first time.  I know that Interspeech organizers have been working to lower the acceptance rate, while it sounds like there has been pressure to keep the size of ICASSP large, even at the expense of a higher acceptance rate.  IEEE (the ICASSP parent organization) is a much larger bureaucracy than ISCA (Interspeech).  There are clear expectations from IEEE about the expected revenue from hosting a conference, which translates to expectations on attendance and therefore the number of accepted papers regardless of the number of submissions.

Despite the near constant rain, I genuinely enjoyed Vancouver and ICASSP 2013.  I'm looking forward to the next.



Monday, October 22, 2012

Reading and Reconnecting

With travel finally settling down for me, but ramping up for my wife's book tour, I'm able to settle in to some long overdue reading, thinking and planning.

Also, after an exciting conversation with Dogan Can, during a trip to USC's SAIL lab, I'm trying to get more on top of sharing ideas, progress and information here.

First up: some drill-down reading from Paul Mineiro's blog post on Bagging!

Ensemble methods work too well for me to understand them so poorly, so:

  • How out-of-bag estimates can be used to get at generalization error (better than cross-validation can).  
  • The relationship between the bias-variance tradeoff and ensemble methods from this lecture.  This is a nicely framed discussion of ensemble methods that I hadn't seen before.

Lying Words: Predicting Deception From Linguistic Styles.  This paper describes a common pattern of language use in deceptive story-telling:  Less self-reference. More negative emotion words. Less cognitive complexity.  

I'm looking forward to verifying these claims on some old deception data. And taking a look at debate transcripts through this lens.