This is a mobile version, full one is here.
Yegor Bugayenko
14 April 2021
How We Organized the First ICCQ
First, let me clarify what kind of conference we are talking about. There are two types: professional and academic. The difference is huge. My understanding is that professional conferences are for practitioners, while academic ones are for researchers. ICCQ, which we organized this year, was an academic conference. I haven’t had any expertise in organizing such things, and had to go through it all for the first time. Here is a more or less detailed description of the journey. Feel free to learn from it and make a better conference yourself. We will try to make a better one next year, ICCQ 2022.
Idea
As I said in my opening speech,
the purpose of the new event was to
help the Russian computer science research community better connect with their worldwide
colleagues. Besides having a good intention I also knew where to get a supportive and reliable sponsor.
The rest was just a matter of implementation.
Name
There were many options to chose from, but it seemed
that the de-facto standard was a few letters, all-caps, like
SPLASH,
ICSE,
PLDI,
PLOP,
etc.
There are expections like EuroSys
or EuroS&P but
this naming convention is much less popular.
We picked ICCQ, which was not yet actively used in Google and
the domain
with the .ru
extension was available. Initially, we aimed for .org
,
but since the event was supposed to always be Russia-based, the .ru
seemed to be a perfect choice.
Subject
Many conferences aim at many topics, maybe in order to
attract more papers. We decided to behave in the opposite manner:
we picked a single topic of code quality and decided to reject all papers,
which would not be relevant to it.
We didn’t even publish the list of topics, like many other
events do. We just said that we are about code quality and
how to increase it.
Organizers
Organizers were the people who I needed most of all.
They were not supposed to write papers or make speeches, but without
them I would not be able to make the conference. Not all of them
stayed with us for a whole year, some of them joined and quit, some
moved to the Progam Committee (PC) later. But I’m very thankful to
Sergey Belov,
Sergey Prokhorov,
and
Jeff Zhu—they helped a lot!
Dates
We picked a date for the event about eight months ahead of us
and very soon realized that it was a mistake: too close.
It should have been at least one year ahead of us. We had to re-schedule.
We also set three important dates:
1) “submission deadline”—four months before the event,
2) “author notification”—two months after the submission,
3) “camera-ready copies”—one month after author notification.
This timeline seems doable, but pretty tight. Next year we’ll give
our PC more time for reviewing: two months is definitely not enough
for a large event.
Website
We made it on GitHub Pages with Jekyll:
the source for our website is here.
We knew that most conferences use some hosted site builders, like
conf.researchr.org, but we decided to swim against the
current and build the site the way we believed was right. No surprise,
we’ve got many complaints about that from people who we were trying to
invite into our team, as partners or co-organizers. They were all starting
the discussion with “You need to do your website right if you want
your conference to be a success.” Nevertheless,
the original design of the website survived. By the way, some of the content
we copied with some modifications from the
SPLASH website.
Host
We were lucky to have a very famous Russian University on our side
right from the start: HSE. We knew people from there
and they were not against hosting the event and putting their name on our website.
My biggest thanks go to
Sergey Zykov,
Sergey Avdoshin,
and
Ivan Arzhantsev.
Venue
HSE was also not against hosting the event physically, on their premises.
The event was not going to be big (up to 100 people attending).
We agreed with HSE that they would give us a big classroom on Saturday.
They didn’t mind.
However, later, due to the COVID-19 epidemic situation, we decided to make
the event fully online. Moving to online didn’t cause any problems for
us, it seems. Moreover, we decided
that the next event, ICCQ 2022, will also be online, no matter what
the situation will be with the virus.
Partners
We figured that in addition to HSE it would be good to have a few other sponsors,
mostly to make us look more serious and reputable (as much as it’s possible
for a young conference). We had good contacts with a few Russian
education organizations and they all agreed to join:
MIPT,
MSU, and
ISP RAS.
We contacted industry companies, and they also agreed:
SberCloud,
Yandex,
and
Kaspersky.
And we also contacted two of the biggest not-for-profit Russian media organizations:
RUSSOFT
and
SECR.
They also agreed.
Of course, it was much harder than it sounds. Some organizations didn’t even reply
to us, some of them rejected our invitations. It was a long process that took about two months
of negotiations. We even created this web page
to explain to them why we wanted them to join us.
PC
This was one of the most challenging parts: how to get together a Program Committee
of people who would review the papers and decide which of them deserve to be
published. First, we invited a few people who we knew personally. Then we decided to try and keep
the PC as diverse as possible in terms of country of origin. We didn’t want to
have only people who were living in Russia. We wanted to build a truly international
PC and also a truly professional one, whatever that means.
Because of this plan, the only option for us was cold calling—we didn’t
know so many people from different places.
We sent about 400 invitation emails to people who had participated in similar conferences worldwide.
As you can see on the website: 30 of them
agreed to join us.
IEEE
This was the biggest challenge: to get a
technical sponsorship
from IEEE.
Our friends from IEEE Russia Section C Chapter helped us
(using this form), and
we got it.
If you want to know more details about this step, tg me,
I will try to explain.
It took about three weeks, and they accepted our request.
They gave us their own “Record Number” and ISBN for the Proceedings.
Keynote
We decided that a good conference must have a good keynote speaker—it is someone
who doesn’t write a paper and doesn’t go through a normal review process.
It must be someone known by the community, in order to make the conference
even more interesting for attendees. We decided to invite
Anders Møller. We personally
knew him from his book recently published online.
We invited him and he agreed! His speech
was definitely one of the best at the conference.
Twitter
We decided to be present on just one social network, since it is the most
popular among tech people and the easiest to maintain:
@iccq_ru. We
tried to post what was going on with the preparation of the event, almost
every week. Some of our authors and PC members followed us.
Registration
We initially thought that the event would be on-site, in the classroom at HSE. That’s why
we created a Meetup page and
collected over 170 registrations. Unfortunately, due to COVID-19, we
had to make
the conference fully online.
Steering
We didn’t know what exactly this was for, but we saw other events doing this.
we formed a Steering Committee of two people from our two most important partners:
Zhang Yuxin,
the CTO of Huawei Cloud and
Yevgeny Kolbin,
the CEO of SberCloud.
EasyChair
There has to be some website to collect papers from authors. We decided to use
EasyChair,
since it’s pretty popular and not as expensive as
some others.
CFP
When everything was ready, there was one remaining question (the biggest one!): how to
collect the papers from authors. We had to find a way to promote the conference so that
researchers decided to submit their work results to us. Aside from using our
own existing Twitter/Facebook/Telegram channels, there were two options:
call-for-papers (CFP) distribution, and paid ads.
We made PDF and
TXT versions
of it and then posted to a few places:
WikiCFP,
call4paper,
AllConferenceCfpAlerts,
SEWORLD,
types-announce,
and
DBWORLD.
Ads
We were lucky enough to have a decent budget for paid ads. We placed
our banners into
Communications of the ACM
and
IEEE Computer.
We also ran some paid ads on Twitter.
Waiting
We were waiting for a few months with almost no result. Some papers
were coming but their quality was obviously pretty low. We were
very nervous, to say the least. We didn’t have any backup plan.
If there were no good papers, I was prepared to call it all off
and admit failure (and accept the monetary losses).
Extension
When the deadline arrived, it was obvious that we hadn’t managed to
collect enough papers. We
decided
to give our authors another two weeks:
the deadline was extended to December 18th.
It was worth it! A few very good papers arrived last and we
phew-ed.
Invited Talk
To make the conference even stronger, we decided to invite someone
who we knew and respected. Just like we did with the Keynote talk,
but this time with a full paper to publish too.
We invited Veselin Raychev,
CTO of Snyk. He wrote
a survey paper
for us and we promised him that it would be fast-track reviewed.
Without such a promise he would probably not have submitted it to us, since
we are too young and small. But since we sort of guaranteed the publication, he agreed.
This is how, I believe, invited talks work.
Desk Reject
We decided
to reject six papers before even sending them to the PC
for review. The most popular reason was: out of scope. Some papers
were about something completely irrelevant. One paper was even auto-generated,
along with some very funny typos.
Bidding
We missed this step, because I didn’t know about it. I would actually
skip it again, in the next conference, but most events do a bidding process,
as I’ve been told. They ask all PC members to go through the list of
all submissions and pick those they want to review. Mostly they will
probably pay attention to the subject, maybe to the quality of paper. In the end,
the distribution of papers among PC members is not randomized.
I don’t like this.
Reviews
I went to EasyChair dashboard and clicked “assign papers to reviewers
automatically” and the system did it for me. We’ve had 17 papers and
30 reviewers. I configured EasyChair to assign at least three reviewers
for each paper. Do the math: most reviewers got two papers, while some of them
just one. However, only about ten reviewers provided their reviews more or
less instantly. Others were not giving us anything and about ten of them
were not even answering my email reminders. We had just seven weeks between
the submission deadline (remember, we extended it by two weeks) and the
day of the final decision. We were not sure that we would be able
to collect all necessary reviews and that’s why we
asked most active reviewers
to review more than just two papers. Surprisingly, most of them agreed to help
us. The bottom line: we managed to collect at least three reviews per
paper, while some papers got five reviews. Because most reviewers performed
their duties on time. Only three reviewers never provided us anything,
without any explanation. 10%. Is this how it should be?
Acceptance
This was a very tough moment: all papers were ranked and we had to pick some number
of them for the final publication (and reject all the others).
The question was: how many out of a total of 17?
We were thinking: Having a conference with a small number of published articles is probably not a good
indicator of success. On the other hand, publishing something that
was explicitly rejected by reviewers would be at least unfair to other
authors, and to the reviewers. We had a very intense discussion and decided
to accept just six papers. Why six? Because the average rank of each of them
was positive. All papers with negative ranks were rejected.
Copyright Transfer
In order to publish our papers in IEEE Xplore, which is very important for
all authors, we had to transfer the ownership of all texts to IEEE: this is
the requirement of IEEE. We built a small online form
for that and asked all accepted authors to fill it in
(the source for the HTML form is
here). They did.
YouTube
I created a new YouTube channel,
verified it,
and scheduled a new live stream.
Agenda
We put all talks on the timeline and published the conference agenda.
Then I created Google Calendar events for all sessions and invited
all speakers to their events. I also scheduled a Zoom meeting
for the day of the event and shared the link via the Google Calendar
events.
Welcome Notes
We asked our Steering Committee Chair and Program Committee Chair
to write short one-page welcome notes. They did
(here
and
here).
I also wrote mine, on behalf of the Organizers:
here.
Final PDFs
We asked all accepted authors to send us so-called “camera-ready copies,”
which were basically their original papers, but formatted exactly
according to our requirements. Lucky for us, all authors were using
LaTeX, so we just gave them a few configuration lines for the source
and all the papers look very similar, formatting-wise.
Proceedings
Then, we had to design the book, binding together all welcome notes
and papers. I did it myself this time and you can see the sources
here.
Xplore
Next we packaged the entire ZIP archive and
uploaded
it to IEEE. We didn’t get any response back. We just crossed our fingers
and started waiting. You can see how the package was rendered in
LaTeX, here. I designed
a collection of scripts (bash + Ruby), which generate all PDFs and
the required
package-list
file for IEEE.
Prints
We found a printing company here in Moscow and ordered them to print
us 100 copies of the proceedings,
in B5 format
(100g/m2 paper,
matte cover). We paid around $10 per copy
and it took about a week to print them all. By the way, we were explicitly told by
IEEE that we are not allowed to sell our Proceedings at all or even
give them away for free, unless it’s only for PC members, authors,
and people closely related to the organization of the event. That’s
why we printed such a limited number of copies.
Zoom
On the day of the event, we started our planned Zoom meeting and
connected it to the YouTube live stream,
here is how.
I clicked “Record” and in six hours there was a 2Gb video file with all presentations.
I also asked all speakers to click “Record” on their side too
and then send me the files Zoom created, in order to have the highest
quality of the video/audio possible.
Video Publishing
I asked a friend of mine to edit the large video file, cutting it into 11
pieces: three short welcome speeches, one 45-minute keynote, one invited talk,
and six half-hour sessions. Then I published
all 11 files
to YouTube.
I also had to create their front images and some text descriptions.
Gifts
We packaged our Proceedings in custom-made boxes together with
small chocolate bars, stickers, and a tourist guide book about Moscow.
We sent them to each author, each PC member, and each partner (about 60 boxes).
What’s left will be used during the year as promotional materials
to advertise our future events and invite new PC members and authors.
Published
We got an email from IEEE notifying us that all papers were
published in IEEE Xplore, here.
Mission complete!
P.S. If you decide to organize your own event, don’t hesitate to let me know, maybe I will be able to help you somehow.