Why we are not using PubPub in NIME 2023

Publication Ecosystem
1

We will not be using PubPub in NIME 2023 for a couple of reasons, which we outline here. I am posting this decision in the forum so that people can comment on it and provide feedback on why we should or shouldn’t continue to use PubPub in future NIMEs.

Mild reasons:
• PubPub’s main advantage is to allow members in the NIME community to comment on papers. However, I have not personally seen this happen in many papers, not even in papers that have received awards.
• The formatting looks odd in comparison to other academic proceedings.

Critical reasons:
• It is somewhat complicated to handover and administer between conference chairs each year (there is not a clear guide on how to do this in the NIME cookbook either).
• It adds extra steps to the proceedings publishing process delaying it significantly (in previous years volunteers worked really hard on exporting all the PubPub links to LaTeX and then to PDF, and had to liaise with all of the authors to properly format their submission’s image files)
• Exporting to LaTeX not always works in PubPub
• Reference management is not supported in PubPub

Severe reasons:
• Though PubPub offers some kind of version control, a site crash can sometimes result in losing your draft and not having access to it until a PubPub support technician restores it for you.
• Web indexing is messy, thus some paper titles are not appearing correctly in Google Scholar, resulting in reference duplications and author omissions.

1 Like
2

This seems sensible. PubPub is promising in concept, but as we found last year, it still has a number of practical limitations that mean extra work for the community to keep up with the basic tasks of producing and archiving each year’s proceedings.

Will NIME 2023 be returning to the pre-2021 LaTeX and Word templates?

We are still looking at options for creating a new journal-length publication venue for NIME; as with PubPub, one goal would be to enable better integration of other forms of media into papers. Whether that platform could become a host for future NIME proceedings is something we could discuss as a community later on.

3

Just to clarify that the two primary motivations for choosing PubPub some years ago were:

  1. Media-rich publications: Given our field’s nature, text and images are insufficient to explain what many NIME researchers are working on adequately. Including audio and video in publications and properly linking other material is key for a modern publication ecosystem.

  2. Accessibility: we have been struggling with improving the NIME archive for several years, including optimizing the quality of all the PDF files. Unfortunately, PDFs are generally not very accessible, particularly not when a myrid different software generates them on various OSes. If we want to take universal design seriously in the community, PDFs are not the way to go.

I agree that PubPub is not perfect. I also have experienced the annoyance of losing an hour’s edits (after which I started writing locally and copying in the content afterwards). But I still think that its benefits outweigh its shortcomings. One important benefit of PubPub is that it actually indexed at all. I have personally been trying to handle the indexing of our old archive manually.

Do you have an alternative solution? I hope you are not thinking about returning to a PDF-based solution (particularly not the very problematic ACM-based templates we used to use!).

Given the short deadline before submissions this year, I suggest staying with PubPub for now. It is important that current chairs work with the previous chairs to sort out issues. As @andrew mentions, we are currently discussing how to set up a NIME journal. As mentioned in some workshops at previous NIMEs, I think we can set up a good publication ecosystem that includes both proceedings and a journal. This is the type of thing that PubPub could support, although there are also other solutions out there. In any case, we should be cautious about changing the core infrastructure of the community too often.

4

@andrew Yes! We’re actually using the same templates as in 2020 for both Paper (for Full, Short, Demo, Workshop, Installation and Doctoral Consortium papers) and Music proceedings. The downloadable templates files can now be found in all of the submission pages.

@alexarje Sorry, yes, we’re going back to PDFs. AFAIK for 2022, the decision was to publish PDFs in the ACM format in the proceedings archive anyways. Though I agree with the good intentions that led to the decision of previously using PubPub I think that the community should have been consulted before transitioning to this format. We in fact tried to use PubPub initially, but decided to revert to the ACM format for simplicity sake and with the aim of providing a familiar template for most authors, within a reasonable timeframe. I’m not personally aware of any alternatives but I don’t find the previous template problematic either (curious to know why it was so).

5

I feel @alexarje’s first point is the key motivation for moving to PubPub (or alternatives). However, at the current PubPub state, I agree that (my) user experience has been tolerable at best. Authors I work with have been using different strategies to overcome difficulties regarding reference management and exporting to different formats, but those workarounds end up consuming time that could be put into writing.

I personally feel more comfortable using LaTeX, but I imagine managing/indexing NIME proceedings or dealing with the quality difference between generated PDFs (looking at you Word template) is also troublesome for the community.

I feel the main problem is jumping between formats each year, especially given the short deadline. Regardless of the chosen format for this year, I suggest keeping the discussion alive with the NIME community to come up with an agreement for future editions.

6

In addition to what I mentioned earlier, there are multiple problems with the old templates:

  1. Quite obvious visual differences between the LaTeX and Word templates. It would help if everyone was “forced” to use the LaTeX template, although I guess that is not realistic.
  2. Poor accessibility of the generated PDFs, including lack of metadata, alt text, etc. In some PDFs, you cannot copy text from individual columns, but end up with text fragments from multiple columns.

Fortunately, we have some scripts around that help solve some of the PDF problems. But it doesn’t solve the indexing issues, which has been a long-lasting pain for an independent community like ours. Zenodo works well as a repository but is not indexed. We discussed moving to OSF which supports “bucket-based” archiving of data and media together with PDFs. It also has DOIs and (I think) is indexed by Google Scholar at least.

In any case, as a progressive community, I definitely think we should continue to push for a solution that goes beyond PDFs in a web folder.

1 Like
7

I think that the community should have been consulted before transitioning to this format.

Well, I suppose we didn’t hold a plebiscite, but there have been many discussions, workshops, surveys, announcements, and invitations to collaborate within the NIME community (to the extent that it can be defined) regarding publication formats. There was a process of trialling pubpub with various folks, both on the 2021 committee and just volunteers from around the community. NIME is a doocracy: the folks that show up and do things tend to end up making the decisions.

PubPub is evidently not perfect, but I think the hope was that it would be at least a good experiment as we keep an eye on other web-first publication technologies (and create them ourselves). It’s disappointing to feel that we are going backwards.

I prefer reading pubpub articles, particularly during the (online) conference and particularly on devices with smaller screens. I really appreciated the timely publication of the 2022 proceedings and the fact that it was indexed quickly.

Clearly there are problems with PDF export for Pubpub which never worked well.

I do see the 2020 template (and earlier) as dated. If we are using a latex template, it would be good to use one that looks a bit more modern. Unfortunately the present ACM template isn’t really appropriate for non-ACM events.

I think I’ve demonstrated with my other posts that it’s possible to use current tools to have both PDF and HTML with non-latex source. If someone wants to pick up that work and get hacking on new solutions, well, the template is CC0…

I guess it would be good to see some efforts here to try to address some of the issues with (pick some) indexing, authoring, reading, accessibility, archiving, DOI minting. By switching back to the old templates, more of these things are kicked down the road to other members of the community.

A few suggestions:

  • kill the word template, document a workflow for importing word into Overleaf and the latex template.
  • collect all source files from authors (no exceptions) for systematic conversion into an HTML proceedings.
2 Likes
8

I guess another point against pubpub for me was reviewing: the PDF output from pub pub that we saw in CMT was never great.

2 Likes
9

I agree with @alexarje that it would be disappointing to deprecate media-rich publications, and negatively impact accessibility by reducing proceedings to PDFs, and the scientific looking two-column format which is unfriendly to the general public. At the same time there is clearly still a lot of work to do to make something like PubPub work, and the details really matter.

Are the issues mentioned really that insurmountable to warrant changing the process again rather than addressing them? Did someone contact PubPub regarding the issues mentioned?

Regarding @charlesmartin’s comment about the “doocracy” aspect, the Paper template officer - NIME Conference Cookbok volunteers have discussed these issues in depth over the years, and there was the NIME Publication Ecosystem Workshop – NIME2020. Keeping in mind these efforts, and general transparency, @juan_pablo it would be beneficial to know what kind of decision-making process was involved here?

Perhaps it’s time for the Paper Template Officer(s) role to be revamped and re-advertised to put more energy into these issues, and directly support conference host organisers each year. It sounds like that was absent from your perspective @juan_pablo, could that have made a difference?

10

It was for transparency sake that I’ve posted our decision here and the underlying reasons. Yes, author omissions are insurmountable (in my opinion, they are in fact harmful).

Most definitely so. As I’ve discussed with @alexarje the absence of a documented process to handover PubPub from previous chairs and not initially knowing who to reach out for (i.e., having to chase for information) doesn’t seem like the most effective way to do things.

As an officer myself I understand the nature of NIME’s voluntary-based efforts as @charlesmartin mentions. However, I believe it’s important to more openly both report on actions (e.g., as we’ve done in the diversity gitbook and in the forum), and have officer meetings more often in which the SC is involved. We have done this in the past, where diversity and eco officers have met to discuss actions with the SC (@alexarje to be precise), but having all officers in a meeting would probably be a good idea.

We probably need a cookbook officer too to centrally document on processes and guidelines for the community, otherwise efforts like the NIME 2020 Publication Ecosystem Workshop remain scattered. The current cookbook is great, but needs a bit of a tweak and updating for transparency and accessibility.

1 Like
11

This gets to the crux of the issue. There’s what we want to achieve in concept, and then there’s the specifics of the implementation.

PubPub offers a forward-looking vision that can be a good match for NIME, including on media integration. For what it’s worth, I looked through the first 20 papers of NIME 2022 and more than half of them embed video or audio (and 2 use embedded code). Comments and multiple releases are mostly unused, and I don’t see a lot of other differences from a standard article. That said, there are some nice interactive features like hovering over the reference numbers to see the citation.

But there are problems too, as @juan_pablo already enumerates, and which I also experienced as a chair last year on the receiving end of plenty of frustrated emails from authors. Yes, we did contact PubPub (several times) and the 2021 chairs did as well. We got support quickly in some cases, slowly in others, and not at all in several cases.

In particular, PubPub support made it clear that things like better-formatted PDFs (or even options for us to customise them) just weren’t on their agenda. Better import/export features and batch processing also weren’t priorities, so we were left with a lot of manual post-processing to make PDF papers that would be recognisable to other communities. I don’t blame PubPub for this; they’re a small team and we’re just one small community amongst many. But we have to look at the platform options we actually have, not the one it might be with more support and resources.

Given a choice of two imperfect options this year, I lean toward PDF templates on the basis that their drawbacks are the same as most other academic conferences, hence familiar to authors and readers alike and unlikely to lead to reputational problems for NIME. But I hope that we can come up with a solution for 2024 onward that retains the best of both worlds, and maybe even more we have yet to think of.

In the meantime I second @charlesmartin’s suggestion to deprecate the Word template and make a well documented Overleaf workflow. This would eliminate 75% of the formatting and accessibility problems right there, especially since LaTeX could be exported to plaintext or other formats for screen reader access.

4 Likes
12

I’d like to make a radical proposal to deprecate the LaTeX template. LaTeX eats a massive amount of time and I think is a huge barrier to participation. I am very experienced with it (I wrote my PhD thesis in it), and concluded that it has a very large number of downsides that I don’t need to enumerate here. I do recognise that with a large amount of work, mentoring and hours of forum-surfing it can produce nice-looking output (not with latex which I find always looks bad, but xelatex output with a nice font can look great), but then I suspect that the reason some people like it so much is because it is difficult to use and they enjoy gatekeeping.

Although I don’t use word myself, certainly removing the word template in favour of the LaTeX one would turn off many people from submitting, and greatly impact the interdisciplinary nature of the conference.

The only forward-thinking option I know of is ‘academic’ markdown processed with pandoc (preferably using xelatex). I’ve run a conference with markdown and word options, take-up was 50-50 and I had no support requests for either, apart from one person who had issues because they’d converted the markdown to LaTeX. That was in 2015, and the markdown tools have only improved since then.

This is what pubpub does under the hood, but with its shortcomings I think it would work better to give clear instructions to people to use it directly. I think the worst thing about pubpub is that it’s open source in theory, but in practice you can’t self-host or develop it, so in practice it is not.

Edit: the conference I refer to is ICLC, and the current hosts are supporting markdown templates as well.

2 Likes
13

The word template is not being removed in this case. We have provided both LaTeX and Word options, as you can see here, for example: NIME 2023 - Call for Music

I 100% agree with the academic markdown idea though.

@Sleitman is actually thinking of assembling a NIME Publication discussion group at some point. So please do shout if you are interested. This also goes for everyone in general who’d like to be involved.

2 Likes