Reflections on evaluating Retro Data call for speakers

What is Retro Data

Retro Data is a new online conference set up by Andy Cutler and myself to focus on the practical application of data technologies. As organisers of two other popular online events, it’s apparent how quickly the latest tech becomes the sole topic of conversation, but that leaves a huge gap for everyone bar the early adopters. We wanted to do something a little different and focus on practical frameworks, fundamental principles and real world experiences. Retro Data was set up to promote talks that aren’t focused on the hype cycle.

We have just finished the evaluation process for this first event and I wanted to share some reflections from it.

The Response Was Incredible

When we first started planning, Andy and I did talk about event size, and we both agreed that we would start with a small, single track event that we could iterate on. We didn’t really know what sort of interest we would get from speakers and we still don’t know what interest we will get from attendees. That means that we have been working towards an event with at the most ten talks.

We received 162 submissions for this alpha event! To give you some indication of how good that is (or how challenging that is going to be for us depending on how you look at it), we receive an average of 170 submissions to DataWeekender for five times the number of slots!

We had our work cut out!

This was the first point that we did discuss again whether we would extend the event. We talked about running for longer, or starting earlier, or even about more tracks but we have resisted change this time round. It is clear however, that speakers have a lot of great talks to offer and so we are already discussing how we might grow from here.

How We Evaluated

We had a first sift to ensure that the talks were relevant for the event. Of the 162 sessions received, about a third were discussing MCPs, Agentic things, Copilot, upcoming features on upcoming product and “in the era of AI”. Early on in the submission process I put in bold letters on the CFS “Before submitting please consider that this event is not about preview features, the hype cycle, or AI.” but they still came flooding in.

For the remaining sessions, we went through a voting process where Andy and I individually set a score against the talks - a simple yes / no / maybe using the Sessionize evaluation capabilities. We haven’t been too prescriptive with our criteria here, but the grading was our personal opinion on whether the sessions would be a good fit for this first event. From there we discounted everything with a combination of no / no or maybe / no which reduced our pool to around 53 sessions.

We still have 53 talks to whittle down to 10! So we did again push on options, do we run a 12 hour event? Do we run two tracks? In the end we decided to stick to the original plan.

For the final stage we did a comparison of similar sessions to reduce the set further - so if two people were talking about a certain widget, which of the two would we prioritise. Of the remaining set we have picked 10 talks on based having to balance coverage of topics and time preferences. Ultimately the driving factor has been most aligned with the idea of the event. Hopefully it will be well received!

Reflections

I’m sure we will have a retrospective post event, but there are a few things we observed and discussed during CFS that we will need to reflect on. And if anyone is reading this and has opinions or thoughts by all means let me know in the comments or directly through contact@retrodata.live.

AM/PM Helped but Made Scheduling Harder

For this event I wanted to experiment with asking speakers when they would prefer to talk. In reality, we are asking you to give up your time to talk for our event, so it seems only fair to consider your preferences for when you would like to speak. Something I observed in DataWeekender is we try to allocate people to time slots based on our understanding of their time zones (we are a UK based event, with global speakers). But that doesn’t account for when a speaker would prefer to speak, just our perception of it.

So I experimented with giving speakers the ability to tell us when they would prefer to speak. Speakers had three options to choose from - AM, PM or both. With AM before 12:00 our time, PM after 12:00 our time. Some speakers chose one, some chose both. What I didn’t really anticipate with this however, was that it would make picking and scheduling talks more complicated. With an event that starts at 08:00 there are only 4 AM slots and so this became an important factor when picking sessions.

We Could Improve the AM/PM Question

We also didn’t have any ground rules for what happens if we pick you but we can’t slot you in for your preferred time? Because of the timeline we have set ourselves for setting up the event, we didn’t afford enough time to reach out to speakers and negotiate if we wanted to, so this time round we stuck to the speaker’s preferences.

We also asked speakers to see into the future! We asked a month ago what their plans were going to be in two months’ time and things can change right? If we do retain this question we probably want to give speakers the opportunity to update their preferences closer to the event and prior to us evaluating.

We Maybe Needed More Clarity on Expectations

I think based on content of some talks that we didn’t provide enough clarity of content expectations. We had sessions about SAP, Salesforce, AWS to name but a few and because we are from the Microsoft community, our reach is mostly the Microsoft community and in turn so is our likely audience. Therefore, we didn’t really anticipate crossing over into other domains. We should have been clearer here.

So for this first event we have stuck with our intention however, there is a discussion to be had about whether we make it more intentional or cover a wider spectrum of content.

We also had a lot of content focused on AI. We felt we made it clear enough that this event wasn’t about the bleeding edge. I even put warnings within the CFS to try to curtail the AI submissions, but they still came. I don’t know if we can do anything about that.

We Really Could Lean More into “Retro”

I don’t know how we would structure this, but we both talked about the fact that we haven’t really gone retro and we want to lean into that more in the future. We absolutely want to encourage practical learning, but some sort of nostalgic talks would be interesting I think, we just don’t know what that looks like right now.

More People Wanted to Talk AM than PM

At least of the final evaluation pool, the ratio of AM to PM was roughly 3:2. And given 60% of our available slots are PM that gave us a bit of an imbalance. If we extended the event, then there would be even more PM slots. This is something we will have to consider for future events. More people want to talk in AM slots so how can we accommodate that?

Based on the response, it might be we run a shorter event but more tracks so we can accommodate more AM sessions. We could also start earlier in future. We will also set more ground rules around what happens if we can’t fit you in your preferred slot.

It hasn’t been perfect this time round, but I think including this question is beneficial for speakers but making it work correctly is just as important. I would love to get feedback on this question if you can afford the time. I believe it is a valuable question to ask, but as a speaker do you?

There is a Lot of Amazing Content

This isn’t just mush to make you feel better because you didn’t get in, there was a lot of seriously good content that we couldn’t fit in. We didn’t really know what to expect but speakers, you didn’t disappoint and I felt in the main you got the premise of what we wanted to get out of this event.

We have had talks on fundamentals and enduring principles across all aspects of the data platform and on reflection trying to condense all of that into 10 sessions and still cover the entire spectrum of things has been extremely difficult. We have discussed the idea that we could easily do a whole day of talks dedicated purely to database, to Power BI etc. Or conversely, we could run multiple tracks with dedicated themes.

Wrapping Up

So there you have it, a very difficult evaluation session but hopefully a useful insight into how things went. If you have any feedback, good or constructive, please reach out and let us know. The schedule will be announced on the 23rd August so stay posted. In the meantime you can join the meetup group and register for the alpha event.