The general layout as I envision it currently can be seen here: https://github.com/SPAAM-workshop/AncientMetagenomeDir
*Thread Reply:* Ok, please make a PR so I can check! I want to be careful that it is just for consistency within this database.
I don't want it to get very large and inflexible, as that is not the purpose of this!
*Thread Reply:* But of course making it as portable as possible would be awesome
*Thread Reply:* The ideal would be to start adapting the standard that is going to be part of GSC and hopefully adopted by INSDC
*Thread Reply:* OK. I'd like to have a look. Maybe most columns are fine with that but then some could be potentially problematic (cultural era, dating etc...), 'cause Archaeologists ๐
*Thread Reply:* Have you already got tihs defined anywhere public?
*Thread Reply:* The repo is awesome, I will try to talk this week with Guy Cochrane (head of ENA) if we can start a mini project to integrate this
*Thread Reply:* Is not open yet, but the core descriptors of mixs are open
*Thread Reply:* Once we are also a bit furhter along, I'll try to add some CI checks, so we can check PRs against the standards.csv file(s)
*Thread Reply:* Those are the ones you should first encourage to comply
*Thread Reply:* Those have to be obligatory in the tsv file
*Thread Reply:* Ok. Yeah, then please make a PR with those proposals and definitions
*Thread Reply:* I will invite you to the doc I am working and then I can do the PR
*Thread Reply:* If doc is google drive: jfy133@gmail.com
*Thread Reply:* Sounds brilliant I would love to join - thanks very much!
*Thread Reply:* Hi James, thanks for including meโ excited to be part of the project! Iโm very much a beginner with git and would love to join in on the intro. Iโm based in Chicago so 14CEST works for me!
*Thread Reply:* thanks James-- Iโm shreyaramachandran
*Thread Reply:* Thanks, I'll be there as Allaby-lab.
*Thread Reply:* Is your name associated with the account? As in the display name?
*Thread Reply:* Just asking as that would be important for you getting correct attribution
*Thread Reply:* Thanks James - I have set up an account under eg715. Looking forward to tomorrow
*Thread Reply:* You should've recieved an invitation to join the organisation!
Review requested for: https://github.com/SPAAM-workshop/AncientMetagenomeDir/pull/21
}
https://hackmd.io/@jfy133/H19kmDalw
*Thread Reply:* Hey Becky, no problem. If you can wait 20 minutes I can go through with oyu privately if you want
*Thread Reply:* Or just message me here if you look through the slides
*Thread Reply:* Yes, I'll have a look and let you know if I have any questions. I hope it went well. Oops!
*Thread Reply:* Surprisingly well!
*Thread Reply:* Whcih is a good sign I hope ๐
*Thread Reply:* Okay, I added Fagernas 2020 and requested your review, but now I'm getting emails saying it has failed tests. Let's discuss when you have time.
*Thread Reply:* Will do! Tests are new, might need tweaking
*Thread Reply:* I just had a look
*Thread Reply:* Try taking out the umlaut on the a
*Thread Reply:* Might be a limitation of the Regex atm
*Thread Reply:* Good shout. I think it's gone through now?
*Thread Reply:* It did indeed!!! :mask_parrot:
*Thread Reply:* I'll do the manual check later!
*Thread Reply:* Either tonight or tomorrow, toddler allowing
For example in here: Fagernรคs2019, is not allowed as it has an invalid character (รค). In his case, make a new commit to just replace it with the character without the accents. This is unfortunately a limitation of Regex (due to a very complicated problem, so this is a compromise)
@channel anyone willing to do reviews for the following for me: https://github.com/SPAAM-workshop/AncientMetagenomeDir/pull/24 (7 samples)
https://github.com/SPAAM-workshop/AncientMetagenomeDir/pull/21 (~40 but most are from the same site)
Basically just need you to check the metadata info looks approximately right (ignore the minor changes)
*Thread Reply:* Thanks James!! ๐ . Sorry for not answering earlier.
*Thread Reply:* No problem. Feel free to ask any questions if you want to catch up
*Thread Reply:* I will reach out to you as soon I finished reading all the comments . Thanks!!
Hi everyone! I have heard about this project and though a study we recently published about historic herbarium metagenomics (samples were collected up to 150 years ago) might be interesting for it although the samples are not ancient. What do you think?
*Thread Reply:* For clarification, I consider anything ancient being from anything not currently living ;)
*Thread Reply:* Yes, I think thatโs the way to go. But I agree we need to limit the number of categories. I think the 4 categories I mentioned cover everything I can think of (for example parchment could go into either microbiome-host or environmental)
*Thread Reply:* Oooh careful there. I wouldn't consider myself a microbiologist ;)
*Thread Reply:* A lot of ancient eDNA focuses on fauna and flora, not microbes
*Thread Reply:* But most of the papers in the PRs are microbiome related
*Thread Reply:* Also environmental ontologies cover non living entities
*Thread Reply:* Only because they are the people in pestering distance from me ;)
*Thread Reply:* But yes, ok
Then for example from EnvO you can go to FoodOn (https://www.nature.com/articles/s41538-018-0032-6)
*Thread Reply:* I updated the file just now, can you review it again?
*Thread Reply:* Will do! Just tweeting aobuti t ๐
*Thread Reply:* (the project not the PR)
*Thread Reply:* Just nee dto fix something. NAs don't work for strings in JSONs unfortunately ๐
*Thread Reply:* I'm using 'Unknown'
*Thread Reply:* Iโll put that in instead then and make a new commit
*Thread Reply:* That's only for strings
*Thread Reply:* LatLon is fine with NA I think
*Thread Reply:* Updated with Unknown for site and geo_loc
*Thread Reply:* Last thing:
Gb1-reg should be Gb1 now. And then we can merge ๐
*Thread Reply:* > One material type Iโm unsure about is burial/grave sediment/soil. This has been argued to still contain traces of the human microbiome. But itโs mostly soil so maybe nevironment fits, but itโs not โnaturalโ if we consider it may have been in a grave with human artefacts (metal, wooden coffin etc). Another one Iโm unsure about is metagneomes derived from seketal material (e.g. from a jaw bone). This will mostly reflect the burial environment but it is skewed to microbial taxa that can colonise bone... For this we should ask Pier, he will give us a good advice
*Thread Reply:* Can you do that?
*Thread Reply:* That's very embarrassing, I've been complaining that loads of gut microbiome papers don't specify that in the title, they just say human micrbobiome. So I get excited and rapidly disappointed when it's all poop.
But clearly I've been sucked in ๐ข
*Thread Reply:* Nothing to be embarrassed, most probably many that use microbiome in the title they refer to the microbiota...
*Thread Reply:* Maybe have to, even if it's ugly
*Thread Reply:* When you are looking at deep seq/permafrost, it's definitely just(/primarily) microbes you're looking at, right?
*Thread Reply:* The whole community
*Thread Reply:* Wondering if we could separate host-microbiome vs envionrmental-metagenome
*Thread Reply:* But also eukayrotic free-DNA?
*Thread Reply:* You have a mix of the different dna pools
*Thread Reply:* Itโs a mess
*Thread Reply:* Yeah, which to me would be metagenome rather than just microbiome
*Thread Reply:* Indeed ๐
*Thread Reply:* But of course same applies to 'host-associated' microbiome
*Thread Reply:* I would have a category as environmental-microbiome or similar as there will be people focused on this, i.e. looking for spore-related studies or recovering frozen microbes/viruses
*Thread Reply:* I would use the definition of microbiome from here: https://microbiomejournal.biomedcentral.com/articles/10.1186/s40168-020-00875-0
*Thread Reply:* I wonder if then we are getting to specific then. Given shotgun contains not just microbes....
maybe ancient-hostassociated-metagenome ancient-hostassociated-pathogen ancient-naturalenvironment-metagenome ancient-anthropogeneic-metagenome
or something
*Thread Reply:* Yep, I think this would be the most general
*Thread Reply:* then we cover all cases and we are not limited to being microbiome
*Thread Reply:* Sediment crew ๐คฃ.I like it. If we ever get money to pay for slack I'll make you a team with that name
Slides: https://hackmd.io/@jfy133/B156Z2mfP
It sounds good to me, we also need to think about the issue @ivelsko raised about pathogens (https://spaam-community.slack.com/archives/C0183TC8B0R/p1597061759074900) maybe something like parasite-associated or similar. I donโt know how much this can be useful https://github.com/GenomicsStandardsConsortium/mixs/issues/65 or https://journals.plos.org/plospathogens/article?id=10.1371/journal.ppat.1008028 for this discussion
}
}
*Thread Reply:* Majander 2020 has a genome of 46.89% covered at 1-fold
*Thread Reply:* OK, then 50% is a bad idea
*Thread Reply:* That indeed is arguably enough
*Thread Reply:* It also depends if you have the correct ref genome to begin with etc.
*Thread Reply:* Again, up to the re-analyser to evaluate ;0
for example this is well charactized in MIGS https://www.nature.com/articles/nbt1360
Hey all, Iโm getting a bit confused about recording the latitude and longitude of the sites. Are you specifically looking up the site location? For example one sample I am looking up now is from Middenbeemster. I have looked up the reference to the site, it is https://www.sciencedirect.com/science/article/pii/S1879981713000831 but I donโt see any more information here about the specific site location. Whatโs the best thing to do?
*Thread Reply:* Try get it as close as possible
*Thread Reply:* Just whilst youโre here James, I am only recording shotgun sequenced data right?
*Thread Reply:* I normally go in the following order:
1) if there is lat/lon given, use that 2) if there is a map, approximate on Google maps 3) if there's is a local town or something, or the site is from a site where there, just put the town name 4) if it's not really clear, put it in the middle of the region but only give 2 decimals
*Thread Reply:* Correct, only shotgun
*Thread Reply:* For middenbeemster you can use the same values as Mann 2018
*Thread Reply:* That is super helpful - thanks! Mind if I copy that to the notes I am writing up?
*Thread Reply:* They are the same samples
*Thread Reply:* Feel free to copy everything I say
*Thread Reply:* Certainly better than my own memory ๐คฃ
*Thread Reply:* Brill thanks! Hahaha, same I have to write EVERYTHING down
*Thread Reply:* Like I said if you could it on the wiki that would be awesome (not obligation though)
*Thread Reply:* Of course, happy to share ๐ Iโm making my way through one then it would be helpful for someone else to cast their eyes over it too
*Thread Reply:* I just merged the current master into your Ziesmer-2015 branch so you don't get too far behind
*Thread Reply:* Great - thanks. Hoping Iโll be done pretty soon will let you know
*Thread Reply:* I agree, it would be useful to have column listing rawdata/consensusonly. If both are provided, do we list just raw_data? it might be useful to know if both are available @aidanva @James Fellows Yates
*Thread Reply:* Suggestions on concise way to code this?
*Thread Reply:* unless we compress
*Thread Reply:* so: raw, consensus, contigs, mag
*Thread Reply:* Ah but then that's not following the suggestions from Antonio ๐ค
*Thread Reply:* but the suggestions from Antonio were dealing with organelles, plasmids and others right?
*Thread Reply:* Oh ffs :face_palm: sorry I'm all mudlded today
*Thread Reply:* correct yes
*Thread Reply:* OK back at it
*Thread Reply:* raw, concensus, contigs, mags
*Thread Reply:* And then combine them as needed
*Thread Reply:* but I think what ash is referring is how we report then the data, so shall we report consensus + raw data for example in the archive_accession
*Thread Reply:* rawconcensuscontigs
*Thread Reply:* But archive_accession should still be at sample level
*Thread Reply:* And that stuff should all be linked at the sample level
*Thread Reply:* it may get messy if people uploaded raw data in one place and contigs in another?
*Thread Reply:* True but that's stil up to the researcher to find
*Thread Reply:* I am just thinking then one will have to mention more than one archive as well
*Thread Reply:* > True but that's stil up to the researcher to find
*Thread Reply:* sure, but wouldnโt that mess up the checks that you do to see if the data complies to the rules?
*Thread Reply:* I don't think so
*Thread Reply:* The archive_accession is currently free text
*Thread Reply:* just checking the code, and it shouldnโt
*Thread Reply:* ok, I think adding the additional column makes sense, and then we agree to have a column with data_type or similar name, with potential fields being: raw, consensus, contigs, mag
*Thread Reply:* how will you define the difference between contigs and mag?
*Thread Reply:* Yeah realised that's the same thing
*Thread Reply:* Or we could just call it assmebly?
*Thread Reply:* I think we always have mags reallyโฆ
*Thread Reply:* yeah, I think assembly is better
*Thread Reply:* And hierarchy: raw < assembly < consensus?
*Thread Reply:* I donโt understand the hierarchyโฆ
*Thread Reply:* assembly and consensus are often used interchangeably in the literature
*Thread Reply:* How does one seaprate a colletion of contigs (an assembly to me) vs a final fasta
*Thread Reply:* if the consensus is called from an assembly, it is sometimes referred to as an assembly.
*Thread Reply:* we could provide definitions of what each means in the context of our db
*Thread Reply:* I woul dopt for that
*Thread Reply:* But maybe you guys can agree on the least ambiguous terms?
*Thread Reply:* Sure, @aidanva shall we set up a singlemicrobe channel for us and others to discuss this stuff?
*Thread Reply:* Sure, letโs be the god of the pathogens, and decide on the terms. Do you wanna set up the channel?
*Thread Reply:* Maybe founders of rather than gods ๐
*Thread Reply:* Please also announce it on #general once made!
*Thread Reply:* This is the thread @James Fellows Yates. Maybe this column is not necessary, but in terms of mixture we could just say โshotgun/captureโ or something like that
*Thread Reply:* Yeah. Ok, but I think my original point still stands
*Thread Reply:* Was thinking ahead, my ba
*Thread Reply:* Letโs no add it now, but maybe something to consider
*Thread Reply:* Looks very nice, I think that would be easily incorporated as a column
*Thread Reply:* OK. @aidanva you can get a 'consensus' on what is allowed to be included, we can add this as well
*Thread Reply:* Although we need to work out how to define if it is a mixture...
*Thread Reply:* if a sample has shotgun and capture data
*Thread Reply:* Remebmer we are listing samples
*Thread Reply:* (as far as possible, of course)
*Thread Reply:* yes, many of the things that can be related to a modern microbiome, most probably is already included in MIxS/MIGS/others
*Thread Reply:* I would use the terms of MIxS and coin our specifics
*Thread Reply:* Sorry ignore me about the mixutre thing
*Thread Reply:* Was on the wrong thread
*Thread Reply:* thinking on wrong thread**
*Thread Reply:* (the list Antonio sent has stuff like: single amplified genome, single-cell, metagenome-assembled etc)
*Thread Reply:* we may have to coin specific terms for our list
*Thread Reply:* but there are terms that we can definitely reuse
*Thread Reply:* I always would try to use one of the existing terms, and if not discuss potential new ones
*Thread Reply:* Yes, just need to be careful we are not using it with different meanings that may mislead people
*Thread Reply:* we donโt want to overload the list
*Thread Reply:* Mm, you shouldn't need run accessions
*Thread Reply:* YOu need the sample accessions
*Thread Reply:* If they are two separate samples (even if from same individual), they can be two lines
*Thread Reply:* Ok great thanks, yeah there are only 2 different sample accessions, but multiple run accessions โ just go with sample accession?
*Thread Reply:* Yes, we only ever use sample ones because run accesisons get messy (that'll be project nr 2 ;))
*Thread Reply:* Hey sorry for all the questions, I am still failing the checks due to it not recognising Borrelia recurrentis. In the list the error is giving Borrelia recurrentis is not listed, but it is present in the enumns/singlegenome_species list (thanks for adding it ๐ ) is it just taking a little time to update? Do you think itโs best to wait for a while or should I remove the PR and make a new one?
*Thread Reply:* yeah, I realised I should do a pull request on main, because that it is where it checks
*Thread Reply:* I asked James to review it for me, so hopefully when it is merged your test can pass and then we can merge your PR
*Thread Reply:* the checks will run automatically once it detects a change
*Thread Reply:* Oh great, in that case sorry I made new commits that werenโt really needed
*Thread Reply:* no worries, I am also figuring this git stuff
*Thread Reply:* btw, I left some comments of changes you should do to the table
*Thread Reply:* not sure if you have seen them
*Thread Reply:* Thanks for the comments Aida I have made the changes ๐ Would you mind explaining the differences in the accession codes please? I was using this link: https://www.ebi.ac.uk/ena/browser/view/SAMEA4771255 I canโt see the ERS2591211 code anywhere here, should I have been looking somewhere else?
*Thread Reply:* @Ele yeah it's annoyingly hidden
*Thread Reply:* I also couldnโt find it at first, but when you are in the project page, you can open the show columns section bar, and you need to tick โsecondarysampleaccessionโ
*Thread Reply:* Top right press 'read files'
*Thread Reply:* Then open the white show column sleection in the middle of the page,
*Thread Reply:* First column, 2/3rds of way down, tick 'saecondarysampleaccession`
*Thread Reply:* Argh! Yep the secondary thing. Got it sorry - this is all quite new for me. I thought I had checked that - maybe I reloaded page!
*Thread Reply:* No worries! I added the new species to the list, and reexecuted the tests so you should see a nice โ๏ธ
*Thread Reply:* Why are we using the secondarysampleaccession @James Fellows Yates?
*Thread Reply:* It's more closely linked to the data itself
*Thread Reply:* The SAMEA-like codes are a part of this BioSample database which I think allows linking to other types of data (transcriptomics, proteomics etc)
*Thread Reply:* @Ele you can merge the pull request now!
*Thread Reply:* But in the the end goal would be to make tools that allow you to automatically download all the sequencing data you request
*Thread Reply:* (all non-UDG treated data from Finland from 1300-1700 year ago)
*Thread Reply:* Or even have automated taxonomic profiles etc which you can download for comparative data
*Thread Reply:* @Ele merge merge! It's an awesome feeling and then to delete the branch ๐
*Thread Reply:* you can also merge @James Fellows Yates
*Thread Reply:* Ele jsut did it
*Thread Reply:* @Ele if you didn't link the PR, please also clsoe the issue (if you've not already done it)
*Thread Reply:* I just clicked and the post arrived which delayed my celebration messages!
*Thread Reply:* Sorry James not really sure what you mean about the linking. I have a prompt to delete the branchโฆ Should I do it?
*Thread Reply:* Linking is separate! But yes, you can delete the branch
*Thread Reply:* As you've added everything from the paper
*Thread Reply:* For the linking:
*Thread Reply:* Wow that feels very satisfying. Thanks both for talking me through ๐
*Thread Reply:* An example from another PR
*Thread Reply:* If you start # (if not at the begining of hte line), it'll give you list of issues and PRs and if you select one, will be rendered as a hyperlink. When you open the PR someone an then quickly find the issue to check for other issues.
A nice trick with github, is if you put certain keywrods like 'close', Github will automagically close the issue for you once the PR is merged
*Thread Reply:* Also once you link it, the issue will be updated and the PR displayed on the issue
*Thread Reply:* (bottom right)
*Thread Reply:* Ahh I see - thanks. And will that # work everywhere? So itโs a way of linking pull requests on the same paper together?
*Thread Reply:* Yes, you can have multiple PRs to one issue, or multiple issues to one PR
*Thread Reply:* I will absolutely be in by the end of next week--life calms down after Monday! Thanks for the timeline update!
*Thread Reply:* Yes, I think so (@aidanva) Have a look at the single-genome slack channel ๐
*Thread Reply:* May I enter the holy grounds, oh god(dess)?
*Thread Reply:* Yes please do add to them - I am bound to have missed something!
*Thread Reply:* This is incredible!! Thank you Ele!!
*Thread Reply:* @Shreya โ thank you! I hope itโs helpful ๐
Release v1.0.0: Ancient Ksour of Ouadane Due by September 04, 2020 Last updated less than a minute ago Release Description First major release of AncientMetagenomeDir of Host associated meta- and single-genomes samples older than 1950. Ancient Ksour of Ouadane, Chinguetti, Tichitt and Oualata: Founded in the 11th and 12th centuries to serve the caravans crossing the Sahara, these trading and religious centres became focal points of Islamic culture. They have managed to preserve an urban fabric that evolved between the 12th and 16th centuries. Typically, houses with patios crowd along narrow streets around a mosque with a square minaret. They illustrate a traditional way of life centred on the nomadic culture of the people of the western Sahara. Description is available under license CC-BY-SA IGO 3.0 from https://whc.unesco.org/en/list/750/
@channel @Maxime Borry has made a much imporved error reporting: (one for @Ele and her Wiki page!)
*Thread Reply:* And no artifacts are saved
*Thread Reply:* Replacing it with | tee -a
*Thread Reply:* But not sure what's wrong with the artefact uploadin. I changed it slightly but maybe calling it wrong
*Thread Reply:* @Maxime Borry artefact is back, and comment always runs, but using | tee removes the fail tick ๐
*Thread Reply:* And also hides the error, which makes the check pass ๐ We donโt want that
*Thread Reply:* yeah that's what I meant
*Thread Reply:* Have an idea, one moment
*Thread Reply:* $ set -o pipefail
*Thread Reply:* @Christina Warinner https://www.ncbi.nlm.nih.gov/bioproject/PRJEB4015
*Thread Reply:* Reverse searching found it
*Thread Reply:* No idea what the original search was doing
*Thread Reply:* I found it - there is a typo in the accession number in the original paper
*Thread Reply:* Okay, silly question. Now I want to add in this data. How do I overwrite these entries? Do I just make a new pull request?
*Thread Reply:* Oh wait - I can see that I can edit my previous pull request. Nevermind!
*Thread Reply:* Okay - I think I did it! ๐:mask_parrot:
*Thread Reply:* I might need to add those species to the list of valid species but if no other errors are reported then it's ready for review
*Thread Reply:* I think I fixed it and now the checks are running
*Thread Reply:* Nope, the checks failed again. It keeps failing on test ancient single microbial genomes
*Thread Reply:* OK, one sec
*Thread Reply:* Might be the webpage it takes the valid lists from hadn't updated
*Thread Reply:* So my pull request has never passed checks, so I just continue to mark it as new rather than a correction, correct?
*Thread Reply:* Oh my bad, I updated the wrong list
*Thread Reply:* No problem! It passed! Now I just need a reviewer ๐
*Thread Reply:* Mmm, no that was a different PR that passed, where you just edited the enum list
*Thread Reply:* In th efuture I 'll put the docs so if you need ot add a new sample host or pathogen you ping one of the core team
*Thread Reply:* Because I realise this is a little complicated
*Thread Reply:* OK! Now tests have passed!
*Thread Reply:* Yes, please ping Ash, Me and Irina in the reviewer box!
*Thread Reply:* I donโt even know what that means!
*Thread Reply:* I think I may have done it
*Thread Reply:* Yes you have perfect ๐
*Thread Reply:* Thank you very much! I might be able to look a tit tonight
And more me (sorry for the SPAAM today ;)).
I'm going to start drafting the paper, please see the following document for my current thoughts on an outline:https://docs.google.com/document/d/1qButPlqSf4YZBv8pYyzyIvQ9fiPL-9YTdJuj4IshV4o/edit?usp=sharing
Thoughts/comments welcome!
To view it while I'm drafting (read: word vomitting): https://www.overleaf.com/read/fjckjfvnqvmx
10 to go until first release!
90% completed manuscript can be seen here for comments already: https://www.overleaf.com/read/fjckjfvnqvmx
I donโt see a โreviewโ button on either version that appears when i click the link, do I need to look at something else? I tried viewing the left panel in Rich format but didnโt see it there, and still have only a โshareโ button when I click the arrows for full screen
What about this one: https://www.overleaf.com/6976312443bkrvfhhfjvfq
*Thread Reply:* I finished making my comments for the pre-print version. Are you planning to include any figures or tables for the pre-print, or will you hold off for the publication b/c of the time crunch now?
*Thread Reply:* Will be generating a couple this afternoon (number of papers over time, and number of published samples) and @ร shild (Ash) is making a map.
If you have any other suggestions let me know!
*Thread Reply:* Have max of 3 objs though IIRC
*Thread Reply:* @ร shild (Ash) youโre not basing the map on the one Tina made, right? I found that one difficult to read
*Thread Reply:* It's awful isn't it ๐
*Thread Reply:* I canโt believe theyโre going to publish it
*Thread Reply:* Is that the one you showed me James?
Anyone up for a quick review of a 2 sample paper?
https://github.com/SPAAM-workshop/AncientMetagenomeDir/pull/237
Only three more PRs to complete and ew can do our first release!
Yes, we had a good surge last week ๐ช
I think we are good for now submissions, as all the papers have been assigned.
The priority would be getting your comments on: https://www.overleaf.com/6976312443bkrvfhhfjvfq
*Thread Reply:* Looks really awesome - thanks for putting all this together. I have added some comments (mainly suggestions to rephrase stuff)
*Thread Reply:* Oops sorry about the missing NHM affiliation, I think I accidently over-wrote when updating the 'leaderboard'. It is in the acknowledgmenets at least. Will re-add it now!
*Thread Reply:* Ahh itโs fine! Youโve got a lot on!!!! Thanks ๐
@channel Are we going to include ancient shotgun RNA studies? for example: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000166
*Thread Reply:* How did they make cDNA? Did they use mammalian polyA tails? If so, then itโs host-specific and not metagenomic (unless you expect to find host-specific viruses in the data)
*Thread Reply:* They generated shotgun RNA data
*Thread Reply:* They also did some metagenomic analyses on the data to look for RNA viruses, but didnโt find anything
*Thread Reply:* I agree, I would leave out RNA for now
*Thread Reply:* But in the paper we have to state that we also have RNA pathogens in the dataset, and not just DNA
*Thread Reply:* Iโm adding it to my comments
*Thread Reply:* But were the RNA viruses ones that integrate into host DNA and would have been DNA sequences?
*Thread Reply:* โLastly, we looked for evidence of viral infection from RNA viruses (both single-stranded RNA [ssRNA] and double-stranded RNA [dsRNA]) in all the sequenced tissues, noting that previous aRNA work has revealed RNA viral genomes in ancient material. We found no evidence of viral sequences in our RNA data.โ
*Thread Reply:* I meant which RNA viruses are included in the database already?
*Thread Reply:* and Influenza A potentially, but we need to discuss it
*Thread Reply:* there will be more aRNA virus papers in the future anyway
*Thread Reply:* I checked the method used for amplification in that paper and in theory it would pick up microbial RNA if it were there
*Thread Reply:* I would still leave out the โtotal RNAโ paper. If you have methods to validate RNA viruses in DNA extractions thatโs a different matter. Ancient metatranscriptomics is still not well-validated in my opinion
*Thread Reply:* OK, then we leave it out. Can always add in a later release if that part of the field develops and becomes more validated
*Thread Reply:* Ok, will give it a go
*Thread Reply:* I think both would work. The workflow is basically a graphical abstract in this case. If thereโs space for two it makes sense for a table to go in that specifically details whatโs being recorded. The figure you sent looks to achieve this ๐
*Thread Reply:* Oh, so then there are two options. this one is 162 words:
*Thread Reply:* Ancient DNA and RNA are valuable data sources for a wide range of disciplines. Within the field of microbial archaeology, the number of published genetic data sets has risen dramatically over the past few years, and tracking this data for re-use is particularly important for studies seeking to reconstruct the phylogenetic histories of pathogen lineages and studies that aim to explain ecological and evolutionary changes in microbial communities through time. AncientMetagenomeDir (archived at \href{https://doi.org/10.5281/zenodo.3980834}{https://doi.org/10.5281/zenodo.3980834}) is an index of published genetic data deriving from ancient microbial samples that provides basic, standardised metadata and accession numbers for rapid data retrieval from online repositories. This list is community-curated and spans multiple sub-disciplines in order to ensure adequate breadth and consensus in metadata definitions, as well as longevity of the database. Internal guidelines and automated checks to facilitate compatibility with established sequence-read archives and term-ontologies ensure consistency and interoperability for future meta-analyses. This collection will also assist in standardising metadata reporting for future ancient metagenomic studies.
*Thread Reply:* And this one is 148: Ancient DNA and RNA are valuable data sources for a wide range of disciplines. Within the field of microbial archaeology, the number of published genetic data sets has risen dramatically in recent years, and tracking this data for re-use is particularly important for large-scale ecological and evolutionary studies of both individual microbial taxa and microbial communities. AncientMetagenomeDir (archived at \href{https://doi.org/10.5281/zenodo.3980834}{https://doi.org/10.5281/zenodo.3980834}) is an index of published genetic data deriving from ancient microbial samples that provides basic, standardised metadata and accession numbers for rapid data retrieval from online repositories. This list is community-curated and spans multiple sub-disciplines in order to ensure adequate breadth and consensus in metadata definitions, as well as longevity of the database. Internal guidelines and automated checks to facilitate compatibility with established sequence-read archives and term-ontologies ensure consistency and interoperability for future meta-analyses. This collection will also assist in standardising metadata reporting for future ancient metagenomic studies.
*Thread Reply:* Mmm problem is the environmental stuff isn't looking at microbes necessarily, they are also looking at wider ecological profile (animals and plants)
*Thread Reply:* Can change microbial archaeology to ancient metagenomics
*Thread Reply:* Yeah, I think the second is better as it's less focused on microbes
*Thread Reply:* (so easier to update)
*Thread Reply:* Okay, how about this: Ancient DNA and RNA are valuable data sources for a wide range of disciplines. Within the field of ancient metagenomics, the number of published genetic data sets has risen dramatically in recent years, and tracking this data for re-use is particularly important for large-scale ecological and evolutionary studies of individual microbial taxa, microbial communities, and metagenomic assemblages. AncientMetagenomeDir (archived at \href{https://doi.org/10.5281/zenodo.3980834}{https://doi.org/10.5281/zenodo.3980834}) is an index of published genetic data deriving from ancient microbial samples that provides basic, standardised metadata and accession numbers for rapid data retrieval from online repositories. This list is community-curated and spans multiple sub-disciplines in order to ensure adequate breadth and consensus in metadata definitions, as well as longevity of the database. Internal guidelines and automated checks to facilitate compatibility with established sequence-read archives and term-ontologies ensure consistency and interoperability for future meta-analyses. This collection will also assist in standardising metadata reporting for future ancient metagenomic studies.
*Thread Reply:* Sounds good to me! Thanks!
*Thread Reply:* You can update the file directly
*Thread Reply:* I am, but just wanted to run this by you too
*Thread Reply:* Okay, all done with editing and commenting!
*Thread Reply:* Intro is now just 28 words over
*Thread Reply:* Thank you very much!
With @James Fellows Yates we have been drafting the following figure for the workflow. Please send us feedback and we will modify it accordingly :-)
*Thread Reply:* Very nice @Antonio Fernandez-Guerra!
*Thread Reply:* Oops yeah ๐ my bad
*Thread Reply:* Great job, @James Fellows Yates!
*Thread Reply:* Itโs so much nicer than the one for the big review
*Thread Reply:* Doesn't have the time aspect though
*Thread Reply:* But you and Ash will work out how to add it in a visually appealing way for the publication
*Thread Reply:* @ร shild (Ash);)
*Thread Reply:* @ivelsko if you are map wizard, please go ahead and work your magic ๐ I will be using the data to test my map making skills, so wonโ be the fastest
*Thread Reply:* No no, I just really really didnโt like the map Tina showed us
*Thread Reply:* Why is the version called โAncient Ksour of Ouadaneโ?
*Thread Reply:* Because it's recommend to give text 'names' too, so I randomly picked stuff from the WHO Cultural Heritage list
*Thread Reply:* To keep it relevent
*Thread Reply:* Oh crap misspelling ๐
*Thread Reply:* Oh, and single genomes are missing
*Thread Reply:* In the release?
*Thread Reply:* Yes, the zip file doesnโt have single genomes
*Thread Reply:* that's odd..
*Thread Reply:* Oh found the issue, here comes the first point release ๐
*Thread Reply:* Can you send me the new link when itโs ready?
*Thread Reply:* Let me know if there are any other issues :face_palm:
Slightly improved version. If you have any comments let us know, and we will do the last edits before adding it to the draft
*Thread Reply:* Those are fine. Proposer sounds a bit odd, but I cant think of another option
*Thread Reply:* These are roles, rather than people, so including curator is OK. (in theory, one person could do all steps, aside from review)
Potential solution, donโt look at the color thoughโฆ
Propose can be the double person.
Contributor a single person.
Curator the 'eye' icon
Auto validation is a computer
*Thread Reply:* Ah I see. I decided to stick with theme_classic because I found it became a bit messy in the other figures (the timeline ones)
*Thread Reply:* i would check how it looks like without the top and side of the rectangle
*Thread Reply:* Looking if I can do that
*Thread Reply:* or add very very light grey for the background
*Thread Reply:* in the main plot
*Thread Reply:* so you have each facet delimited without the need of adding the y-axes and then a minimal facet strip
*Thread Reply:* @aidanva @Antonio Fernandez-Guerra
*Thread Reply:* I can remove the strip titles
*Thread Reply:* Actually no, I don't like this as getting the legend to not be clipped sucks
*Thread Reply:* Eek, ok. Might make it very wide but let me try
*Thread Reply:* yeah it mad eit too wide
*Thread Reply:* And caused the legend to be cut off
*Thread Reply:* I liked it more that way though
*Thread Reply:* but ok, I will remove the legend for the colours
*Thread Reply:* ignore me, since you are going for the long version my comments have no sense
*Thread Reply:* The auto-validation is out of line in the legend
*Thread Reply:* yep, I would like to have suggestions where to place the legend, not 100% convinced
*Thread Reply:* I think position is OK actually
*Thread Reply:* You could put a box around it though
Use this ones better, the dark grey text in the boxes looked bad
https://www.biorxiv.org/content/10.1101/2020.09.02.279570v1 @channel preprint is out!!!
*Thread Reply:* I agree! I think thatโs useful info to have anywayโฆ these databases can be a little daunting for newbies so more info canโt hurt
*Thread Reply:* By negative controls I assume you mean samples that are chosen to act as negative controls rather than laboratory neg controls (extraction and library blanks), right?
*Thread Reply:* I've not seen really the former
*Thread Reply:* I had a soil sample in the salmonella paper that we used as an environmental neg control. It should probably be added
*Thread Reply:* Mmm for the list though we only report retrievable genomes
*Thread Reply:* I might be remembering incorrectly, but doesnโt is say somewhere on ENA/SRA that they donโt want lab blanks to be uploaded? Maybe Iโm completely wrong in thinking this
*Thread Reply:* A lot of papers don't report negative samples so might not be worth it.
*Thread Reply:* Anyway, I think itโs a good idea to include them, if the data is available
*Thread Reply:* Mmm, I've never seen that. I asked them about which taxon to put when uploading my blanks and they gave me a hacky workaround that exists and said they are trying to improve that.
*Thread Reply:* Great! ๐ thatโs good to know
*Thread Reply:* > Anyway, I think it's a good idea to include them, if the data is available
Eek, this might be a bit late now. But honestly I think the same thing goes also for when looking data itself, you should still check the publication's themselves for all data and metadata
*Thread Reply:* The Dir should just act as a pointer to relevant papers, ultimately ๐
*Thread Reply:* Ok ๐ could consider adding a column later where is says if blanks were presented or not, and/or submitted to ENA/SRA. But this would be mostly relevant if we decide to comment on quality in the future
*Thread Reply:* My idea would be that with a future ERR/SRR list we would have a 'blank' category, indeed
*Thread Reply:* One for @Maxime Borry , which should work this time ๐
Side note about Viviane, she has her own wiki page! https://en.wikipedia.org/wiki/Viviane_Slon
*Thread Reply:* Do you know where she still is?
*Thread Reply:* Ah she's back in Germany
*Thread Reply:* Yes, at the Charite in Berlin I think
*Thread Reply:* Say hi ๐
*Thread Reply:* I said hi to Mikkel this morning
*Thread Reply:* You're a friendly bunch. I'm sitting all alone ๐
*Thread Reply:* Not now though
*Thread Reply:* Would be kind of strang ehe is now in my living room ๐
*Thread Reply:* I said hi outside the museum on my way to the lab, socially distanced
*Thread Reply:* Given that you're all on the same papers all the time it seems like you do live together ๐
*Thread Reply:* Itโs such a funky green! :partyparrot:
*Thread Reply:* It's fucking awful ๐ It's a really old template but the only one I could fnd
*Thread Reply:* For number 1, I want to emphasise finding because it's in FAIR. I realise there was a missing word, does it still read wierd/
"While the field of palaeogenetics is recognized as a leader in making raw sequencing data available in public archives, deposition does not always follow FAIR principles with the finding of data for re-use or comparative analyses not being trivial."
*Thread Reply:* Better, can you think of another way to phrase โnot being trivialโ, how about โis not trivialโ or โis not always trivialโ. โBeingโ reads a bit passively to me if I am nit-picking
*Thread Reply:* I am struggling to get it to work without 'being' because of grammar...
"ata must be balanced with the protection of precious cultural and natural heritage. While the field of palaeogenetics is recognized as a leader in making raw sequencing data available in public archives, deposition does not always follow FAIR principles, with the finding of relevant data for re-use often being difficult to do"
*Thread Reply:* "While the field of palaeogenetics is recognized as a leader in making raw sequencing data available in public archives, the finding of relevant data for re-use is often highly time consuming, as deposition does not always follow FAIR principles."
*Thread Reply:* I will change this sentence: โof which are decided upon through consensus of curators and ensure consistency through out the listsโ to โof which are decided upon consensus of curators and ensure consistency throughout the listsโ. You use the word through in that sentence and the one that follows 3 times
*Thread Reply:* Fixed it and replaced other 'throughs'
Final Map and sample age timeline (thanks to @Antonio Fernandez-Guerra for thta!)
*Thread Reply:* It doessn't work for the other plots because they are too close to each other
*Thread Reply:* And so I would rather have it consistent
*Thread Reply:* I think the problem is that the factors don't work properly as you add three different bar plots
*Thread Reply:* I am in my phone now, but in https://github.com/SPAAM-community/AncientMetagenomeDir/blob/master/assets/analysis/ancientmetagenomedir-analysis-notebook.Rmd#L403 add breaks = names(dir_colours)
Couldnโt test it, but I believe it should do the job
*Thread Reply:* The values bit?
*Thread Reply:* you set the order of the legend keys by setting the breaks
*Thread Reply:* Didn't know that! Thanks!
*Thread Reply:* One can always remove a decimal to obscure a siteโs exact location
*Thread Reply:* yeah, thatโs what I was proposing
*Thread Reply:* "watch this space"
Another (old) paper for team PathoPeepz: https://github.com/SPAAM-community/AncientMetagenomeDir/issues/285
Ancient viruses from Japanese teeth: Ancient viruses from Japanese teeth? (maybe one for @Maria Spyrou?)
https://github.com/SPAAM-community/AncientMetagenomeDir/issues/286
@Miriam Bravo I'm going to auto-assign you to your paper (https://github.com/SPAAM-community/AncientMetagenomeDir/issues/296) for obvious reasons, but you can un-assign yourself if yo udon't have time
Hi Everyone! I'm new to the group but have been lurking on the github for a while. I thought of a few opportunities where I might be of some help (Github Issues Link). Just wanted to get acquainted and caught up with all the amazing work in this group:
- SRA metadata. I've got a tool over at NCBImeta that pulls and organizes metadata from NCBI (bioproject accession, biosample info, SRA sequencing depth, etc.). If this tool isn't a good fit, I have a lot of experience working with NCBI's API and parsing their XML format...
- Databases are a passion of mine. Text files are amazing for wide-reaching accessibility, but if there's talk of scaling up, I could help with SQL work.
- Geocoding. I've got a project right now where I'm curating and 'standardizing' the geospatial information and geocoding it (programmitically with GeoPy). Cheers, and nice to meet everyone!
*Thread Reply:* Does the community have a preference for where everyone gets their FASTQ files from (ex. NCBI, ENA, etc)?
*Thread Reply:* IIRC most uploads are to ENA (at least for microbiome) and I share a hatred with a few people of fastq-dump and the SRA file format
*Thread Reply:* But good question. We can make a poll
*Thread Reply:* Yes fastq-dump causes me a disproportionate amount of project frustration...
*Thread Reply:* I would want to know:
- which FASTQ files are from the same library
- what damage treatment (if any) was used
- what library-build enzyme(s) was used
- what sequencer they were equenced on
- how many reads
*Thread Reply:* -read length -batches (extraction, library building, pooling, if different) -indexing strategy (are there potentially internal indexes that need to be removed?)
*Thread Reply:* -DNA molecules per uL of the library (not the pool, for decontam)
*Thread Reply:* โข Library layout (paired-end, single-end)
*Thread Reply:* โข (Basically all the fields needed for the tsv input of nf-core/eager ๐)
Maybe also: https://link.springer.com/article/10.1007/s00334-020-00805-y but I can't access it
*Thread Reply:* Hey, Yup should have some time over the next day or so ๐
@channel
Revised manuscript: https://sharelatex.gwdg.de/1575121292kpsgdpkzgrbb Cover letter:https://sharelatex.gwdg.de/4133753148fhprtsbwvnxh
Please note the follow
- this is a 'locally' hosted version of overleaf that allows track changes
- please do not accept/reject changes (so others can also see it), but please leave or modify the change comments if you are not happy with it (I'm sure @Becky Cribdon will find many problems with my english :P).
Please let me know once you've read the manuscript and you think the changes are sufficient or you have any major issues.
Feedback on the cover letter would be also welcome - in particular the section on the Radiocarbon dating!
EDIT: Please also leave your name in any comments you may leave!
*Thread Reply:* When do you need the comments by?
*Thread Reply:* Friday ideall
*Thread Reply:* (see email)
*Thread Reply:* Finished adding in my (very minor, mostly typo correction) edits in track changes. I noticed that Query 23 does not yet have a response - Iโm happy with either option (keeping as is or making the terms the same). Nice job everyone, and especially @James Fellows Yates ! :mask_parrot:
*Thread Reply:* Thanks very much @Christina Warinner! For query 23 just waiting on Antonio to update the image with the new terms in the text :)
*Thread Reply:* paper: https://royalsocietypublishing.org/doi/10.1098/rstb.2019.0569#d1e1863
*Thread Reply:* data: https://www.ebi.ac.uk/ena/browser/view/PRJEB35426
*Thread Reply:* Also wonโt be available for the next hour or so - I promise I am not ignoring anyone and Iโd be really grateful for any tips! ๐
*Thread Reply:* What sort of metagenomes ๐ค
*Thread Reply:* Do you mean they are just listed in the ENA as metagenomes? Or do they actually analyse in the paper the data as 'metagenome'
*Thread Reply:* Then just list them as single genomes
*Thread Reply:* Technically all aDNA are metagenomes which is why people put the but if they've not characterised the metagenome as a whole we can ignore
*Thread Reply:* Great - thank you! Sorry for the confusion.
*Thread Reply:* Yup they got a low coverage genome iirc
*Thread Reply:* Only list samples they yielded genomes from though
*Thread Reply:* Hey @James Fellows Yates another questions ๐ This paper has uploaded 4 separate libraries for a sample. So the same sample has 4 different sample accessions. Which do I pick?
*Thread Reply:* All four, you can check other examples: we use a comma separated list
*Thread Reply:* In the wiki I have helpfully noted โCareful:ย You may have multiple ERS/SRS codes per sample as some people upload each library as a different โsampleโ. โ but not how to tackle it!
*Thread Reply:* ERRXXXX,ERRXXXXX,ERRXXXX
*Thread Reply:* I'm late to the game, but @Ele , I struggled with the questions too (like James said..all aDNA are technically metagenomes..so I was documenting everything. woops)!
Could someone please merge โAdd files via uploadโ with the master. I have uploaded a few more screenshots to update the wiki ๐ thank you!
}
*Thread Reply:* Done blindly from my phone ๐
*Thread Reply:* HahahBrilliant! Thanks James - wiki is updated and images up to speed ๐
*Thread Reply:* Thanks for the link, but I'm struggling to log in to ShareLatex... it wants a single sign-on from academiccloud.de?
*Thread Reply:* I have an Overleaf/ShareLatex account already but it's not crossing over
*Thread Reply:* And IT teams wonder why everyone goes to the 'non-data protected' variants of everything
*Thread Reply:* Ignore it then (deleted the message)
*Thread Reply:* I read through the changes you made and I donโt have anything to add to them or to change
*Thread Reply:* They do for me as I could log in with my old mpi account, but yes, they wonโt work for non-mpi people ๐
*Thread Reply:* @Zandra Fagernรคs @aidanva you'll appreciate that one ๐
If you want to see the agreed license
*Thread Reply:* Will merge in a day or so. A new shiny functionality will be merged and I want to test it on your PR ;)
@channel new functionality: https://github.com/SPAAM-community/AncientMetagenomeDir#current-status
With each new merge to master the images on the link above will automatically be updated. I need to do some tweaking to them (or @Maxime Borry is going to push them to ultra-fancy), but you get the idea.
The images, as with all other content are under CC-BY so you can take them for presentations/grants/papers/reviews etc (juts need to cite!)
*Thread Reply:* Thanks to @Ele for allowing me to mess with her PR to get it working ๐
*Thread Reply:* thanks James!! done with my teaching for this term so Iโm ready to help push this milestone through!
*Thread Reply:* @Shreya feel free to re-request me for review on the PR once you've commited the changes ๐
*Thread Reply:* Approved and merged, thakn you!
*Thread Reply:* Wooo!!! Thanks James! On to the next!
*Thread Reply:* Both merged! thank you very much!
*Thread Reply:* I'll make the PR during my day ๐ .
*Thread Reply:* Don't feel obliged though of course! It's no problem to de-assign yourself.
Hope thesis writing is going well!
*Thread Reply:* Thanks James, is going well, almost done! ๐
*Thread Reply:* Let me know if you have any problems with the PR btw!
*Thread Reply:* Iโm interested! I speak English and Spanish
I've made a Doodle for the new year to plan about the blog post:
https://doodle.com/poll/vbd3ygyv3s3aypuy?utm_source=poll&utm_medium=link
I should've sent emails to the people have shown interest already, but if you would still like to contribute fill in your avaliability โ๏ธ
*Thread Reply:* If you're happy, please merge directly
*Thread Reply:* Thank you! โค๏ธ
https://natureecoevocommunity.nature.com/posts/29300-enteric-fever-in-sixteenth-century-mexico
https://researchdata.springernature.com/
*Thread Reply:* shreyarama42@gmail.com โ thanks!
*Thread Reply:* Is there a link? or will you send it in an email?
*Thread Reply:* Same Google Doc as before
*Thread Reply:* Ok after running it through my head in a dog walk I realised in particularly not happy with the first half. Too rambly, so I'll probably try that again at some point. But you guys can already start working on it already anyway. Maybe someone else does a better job
*Thread Reply:* Donโt worry about it! No one is judging, itโs good that we have something to work with now ๐
*Thread Reply:* Oh I know. I just meant it might change a lot suddenly ๐
*Thread Reply:* ok cut down at least 100 words from teh beginning. Now I think the last sectoin is too waffly but I'll see what you guys thing
*Thread Reply:* Iโve taken a first pass over the blog post draft, keep what you like, discard what you donโt ;)
@ivelsko and @aidanva / @ร
shild (Ash) it would be great if you guys could double check the PR below (randomly pick a few samples, and check the PRJ code is correct against the ERS/SRS code for the hostassociated-metagenome and hostassociated-singlegenome tables respectively (@Becky Cribdon has already checked the environmental one ๐ช )
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/350
@channel I'm planning on presenting AncientMetagenomeDir at the Wellcome Trust "Ancient Biomolecules of Plants, Animals, and Microbes" (virtual) conference happening in March 2021.
https://coursesandconferences.wellcomegenomecampus.org/our-events/ancientbiomolecules21/
I have written an abstract with all co-authors of the manuscript as 'co-presenters'. Comments and suggestions are warmly accepted here: https://docs.google.com/document/d/1HvM08gpt5YI8fjfr8XE0dbye35ixYQECb8ulxYh2Uy0/edit?usp=sharing
*Thread Reply:* I bet our press department would translate it to German, we could ask them
*Thread Reply:* Oh good point
*Thread Reply:* Can keep in mind
*Thread Reply:* I can also translate it into German, no problem.
*Thread Reply:* Woooho thanks!
*Thread Reply:* Iโve made some feedback! Are we still way over the word limit? I can try and copy it into a separate doc and cut it down if that would help
*Thread Reply:* Yes we are, at least based on the original request. If you can make an attempt we can have a look!
*Thread Reply:* You could also just copy and paste the whole thing onto a new page (with a new header)
*Thread Reply:* Instead of a different doc
*Thread Reply:* Ah yes, good call! Letโs see what I can do
*Thread Reply:* Original request was 400-500, but I feel 600-650 might be ok
*Thread Reply:* Iโm at 660-- please take a look when you get a chance (and feel free to accept/reject my suggestions as you wish!)
*Thread Reply:* @Shreya I have made a few suggestions to shorten it a bit more
*Thread Reply:* Could you have a look, if so we can adopt that version ๐, you did a very good job = thank yu!
*Thread Reply:* OK cool, please accept the changes ๐
*Thread Reply:* (if you haven't already)
*Thread Reply:* done! And made one suggestion on where to spell out FAIR
*Thread Reply:* Ok, moved your version to the top
*Thread Reply:* thanks James! Iโve written to the Spanish team so we can make a plan!
*Thread Reply:* Organisation ๐ช๐ช๐ช I love to see it!!!!
*Thread Reply:* We can't edit the blog post I think. But maybe Norwegian isn't so crucial for impact ๐
*Thread Reply:* But we can see how long the other translations are taking
*Thread Reply:* Yes, I think the five Norwegians who do aDNA all speak excellent english, so impact will be low.
*Thread Reply:* I might give it a miss if thatโs ok
*Thread Reply:* Also, havenโt written in Norwegian much since I was 14, so it would take me a whole day to get this done, which I just donโt have atm ๐
@channel Publication date of the paper - 26th Jan!
*Thread Reply:* I'm honestly too tired atm ๐, if you would be willing to do so that would be great! I'm also not sure if ti's something that would be interesting for them ๐ค
Reminder for the blogpost team: https://docs.google.com/document/d/1YxmFZ3IuQHZInuKLnKMzTQco34NOYwZL6a9_40VAoFA/edit?usp=sharing you can insert your translations in that document! (thanks already to @Maxime Borry and @Alex Hรผbner)
*Thread Reply:* Woo thank you very much @Shreya and @aidanva as well!
https://www.nature.com/articles/s41597-021-00816-y
Ok they did it for us: https://twitter.com/ScientificData/status/1354059031883747330?s=19
}
Twitter
@Antonio Fernandez-Guerra @aidanva @ร shild (Ash) @Miriam Bravo @Maxime Borry @Shreya @Valentina Zaro @Alex Hรผbner REMINDER Please register on https://researchdata.springernature.com/ so I Can add you to a contributor to the blog post!!!!! It will go up next Tuesday!
*Thread Reply:* Do I need to do anything after I registered ?
*Thread Reply:* not for the itme bein
*Thread Reply:* I should be able to add you
*Thread Reply:* Let me know if it comes up in your posts thing
*Thread Reply:* Registered there as well
*Thread Reply:* Haven't seen my posts anywhere
*Thread Reply:* Click on your face > My Posts
*Thread Reply:* Nothing listed?
*Thread Reply:* Done! (with my gmail: shreyarama42@gmail.com)
*Thread Reply:* Found you! Just @aidanva and @Miriam Bravo to go and we will be pretty much ready ๐
*Thread Reply:* I registered with twarinner@gmail.com
*Thread Reply:* I registered too (bravolomiriam@gmail.com)
*Thread Reply:* After your pestering @James Fellows Yates, I also registered
Thanks for the much more interesting tweets that my ones @ร shild (Ash)!!!! https://twitter.com/AshAshild/status/1355200941448974338?s=19 ๐for anyone else who wants to support the word
}
Twitter
*Thread Reply:* โUnfortunately you are not authorized to view that pageโ but perhaps I will be tomorrow!!! Excited for the big multilingual reveal!!
*Thread Reply:* Yeah, I couldn't find a way to add anyone else as 'editors' unfortunately
*Thread Reply:* gotcha! Well I will look out for it!
*Thread Reply:* Thanks @Miriam Bravo!
*Thread Reply:* I don't have twitter but I can send here the translation if anyone wants to tweet in Italian! ๐
*Thread Reply:* @Valentina Zaro if you send it to me here I can tweet it tomorrow!
*Thread Reply:* Sei curioso di scoprire come in soli due mesi (e nel mezzo di una pandemia!) รจ stato possibile passare dall'idea alla pubblicazione di un database di riferimento per il DNA antico? Dai un'occhiata al breve "dietro le quinte" della nostra esperienza con AncientMetagenomeDir!
@Valentina Zaro you need to make it 19 characters shorter ๐
https://twitter.com/jfy133/status/1357276243310350336
}
Twitter
*Thread Reply:* That... could actually be fantastic.
*Thread Reply:* I shouldn't... buttttt
*Thread Reply:* Why not? It would be eye-catching, and would lend itself to presenting several connected concepts around a central idea. That would work well for an introduction to something.
*Thread Reply:* FINE! I'll try and inevitablly waste my evenings on it ๐
*Thread Reply:* But I ain't the one making the poster so ๐
*Thread Reply:* Are you familiar with this person? Probably to blame for my liking of simple but outlandish posters: https://twitter.com/Better_Posters
*Thread Reply:* That person who does the big title/core result in big bold right in the middle?
*Thread Reply:* Ah, that's Mike Morrison. Better Posters did a critique of the Morrison poster but also consider other approaches: http://betterposters.blogspot.com/2019/04/critique-morrison-billboard-poster.html
@channel we have 2 more issues open and we can do a March release, if anyone is up for it?
https://github.com/SPAAM-community/AncientMetagenomeDir/issues/365 https://github.com/SPAAM-community/AncientMetagenomeDir/issues/364
I've made a first(!!!) draft of the poster for the Ancient Biomolecules conference thingy.
Thoughts/feedback welcome!
(Couldn't get a circular design to work unfortunately, but decided to try out an 'infographic' style instead for funsies. Also: before the perfectionists arrive, I know things are off-set or not aligned ;), that will be fixed later!)
*Thread Reply:* You're welcome. By the way, which software did you use to make your poster?
*Thread Reply:* https://inkscape.org/
*Thread Reply:* Icons from openmoji.org
*Thread Reply:* Or drawn by @Antonio Fernandez-Guerra
*Thread Reply:* @Antonio Fernandez-Guerra where can I find more info about drawn?
*Thread Reply:* The icons we used for the workflow are from here https://primer.style
*Thread Reply:* https://octicons-primer.vercel.app/octicons/
If anyone has a spare 3 seconds to approve this: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/371
}
*Thread Reply:* Ok Aida got there already ๐ thanks though!
Ok new version does this help?
Ok, slightly improved version
*Thread Reply:* Is the question mark at the last point of the 'needs improving' section necessary?
*Thread Reply:* Yeah because it raises it the quesiton if other fields would be interested in that or not...
*Thread Reply:* We would do it ourselves (hopefully)
*Thread Reply:* but question is if we should already involved other firleds
*Thread Reply:* Lag:Long @James Fellows Yates
Last PR for today's release ('cause I totally forgot)!
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/378
@channel So as a part of PhD theses as FSU Jena, we apparently have to specify 'how much' we contributed to various aspects of each paper.
I have no idea how one is expected to quantify most of these columns, but I've made up some semi-random BS to make this work - if any of you have an issue with any of the assigned values in the table below, please PM and I can modify accordingly. The only one that has any resemblance to reality is the Experiment (i.e. generating the data) which I partially based on GitHub stats (but then tweaked so I can round up to whole numbers). The rest I have no clue e.g. I have no idea how much each person contributed to the paper writing)
Just wanted to give everyone the opportunity to have their say and that it doesn't appear in their google scholar alerts in a few months time with no warning, and I upset anyone by them thinking I undervalued their contribution!
If anyone has a spare 20 minutes:
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/386 https://github.com/SPAAM-community/AncientMetagenomeDir/pull/385
@channel I've now worked out how to partially automate extracting library level metadata from the ENA/SRA. We now need to work out exactly what metadata would be useful in such a sheet, which would also include all the FTP links to each library.
I've made a github issue here, please see my suggestions, and then leave comments what else we should also add, and we can start making column specifications ๐
https://github.com/SPAAM-community/AncientMetagenomeDir/issues/388
*Thread Reply:* on it! will probably only get to it when the europe squad is asleep though!
*Thread Reply:* No problem !
*Thread Reply:* I can help out! I'll just have to remind myself how to do everything because I've been naughty and not checked in recently (good thing there are awesome tutorials ๐).
OK I think I missed a trick there ๐ (@Maria Spyrou you should name stuff in the future :D)
Random weekend fun fact: ancient host associated metagenome libraries are typically sequenced to 4.8 million reads.
@aidanva @ร shild (Ash) @Miriam Bravo reminder of end of this week deadline so we can make the june release end of this month!
Also, you guys can also help: https://link.springer.com/article/10.1007/s12520-021-01350-z Y/N for inclusion? can't really tell if it's sufficient...
*Thread Reply:* I will work on my PR today. About the paper, I would not include it, they don't have enough reads and they even say that they can not authenticate the aDNA... but that's just my opinion
*Thread Reply:* I will do my PR today as well
*Thread Reply:* Ok we leave it off then ๐ thanks!
*Thread Reply:* Given that this is the coverage they have across the mtDNA after capture, I definitely think we should not include it (from their supplementary).
*Thread Reply:* Tha'ts what I thogught ๐
*Thread Reply:* No problem - good luck with the corrections and your future endeavours! Thanks for all your contributions and support โค๏ธ
*Thread Reply:* Best of luck for the future, @Becky Cribdon!
*Thread Reply:* OK, that's three yes's - gonna change it now!
*Thread Reply:* Requested you all as reviews โค๏ธ
*Thread Reply:* Btw, Zavala is no go
*Thread Reply:* " the raw data for each mammalian mtDNA and human mtDNA enriched library are available in the European Nucleotide Archive under accession number PRJEB44036. "
Also can anyone quickly approve tihs: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/410
*Thread Reply:* @aidanva one more: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/411/files
*Thread Reply:* Forgot the host species ๐คฆ
https://twitter.com/jfy133/status/1409485777172676611
Twitter
*Thread Reply:* I think itโs worth a shot! Itโs only a (max) two page application. Happy to help out!
*Thread Reply:* Will send you doodle invite!
*Thread Reply:* Let me know if you got it @ร shild (Ash), send to your gmail
And still waiting for a review on this minor typo correction: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/416/files
*Thread Reply:* Thanks @ivelsko
https://www.ncbi.nlm.nih.gov/labs/pmc/articles/PMC8278039/
y/n?
@channel Aida, Irina, Maxime and I have drafted our application to the open data award thing. We would welcome your comments/feedback (it's only 2.5 pages) as we would need to submit this at the *end of this week*.
Feedback from anyone with experience with grant writing (e.g. @Christina Warinner) would be particularly useful as us 4 have not done that, but comments from everyone will help a lot! @Hannes I've made a reference to maybe helping put the data into AncientGenomes.com, if you're happy with us saying that?
*Important* we are over the limit by 0.5 pages, so places to cut superfluous stuff would be welcome!
https://docs.google.com/document/d/1edxlM4dxdX2vcuCWhFiNyxu55lfubSjRHVoOfTl6mT4/edit?usp=sharing
*Thread Reply:* Is this the attempt to download the sample/seq. run information from ENA/NCBI that contains all the information on each uploaded library (read count, library type, FastQ/Aspera download URLs)? If yes, I would be interested in.
*Thread Reply:* Yup, but even better: in addition polyermase/UDG treatment etc. which can tell you about levels of damaage etc.
*Thread Reply:* Hi @James Fellows Yates. Sounds great and I'd also be interested though unfortunately won't make spaam3 today due to attendance at another conference.
*Thread Reply:* Ok, I don't know yet when we will meet but can let you know when it happens and you could join if you're free.
I can also summarise what we discuss in the end.
The main thing for me would be finding out what would be good incentives for people to contribute time in cleaning up the data
*Thread Reply:* I'll try to get through some next week!
*Thread Reply:* Oh it's not ready yet ๐
*Thread Reply:* the new structure/raw data is on my fork at the moment
*Thread Reply:* But once we have the volunteers we can arrange integrating the new version into the main repository ๐
*Thread Reply:* I mean I love a good sticker but Iโm also just happy to power through 10 papers
*Thread Reply:* I work for stickers....and pizza....no stickers...no pizza....STICKERS!
*Thread Reply:* idea: what if editors hopped on zoom one day and did a table-a-thon with a couple breaks for games
*Thread Reply:* I was thinking EXACTLY along the same lines ๐
*Thread Reply:* Is that something that your supervisors/PI would agree to?
*Thread Reply:* Mine at least is very supportive of my SPAAMy activities! And I doubt itโd need more than 1 day of working hours
*Thread Reply:* You're already included @aidanva ๐
*Thread Reply:* I would be interested but I donโt know yet when Iโm available.
*Thread Reply:* Just fill in the doodle, you should have optin of 'maybe'
*Thread Reply:* Should have an invite ๐
*Thread Reply:* Or go: https://github.com/SPAAM-community
*Thread Reply:* Pretty sure you type quicker than I am able to think ๐
*Thread Reply:* I'm not even that fast ๐ฑ
*Thread Reply:* You should have an invite :)
*Thread Reply:* Hey James, happy to help out if you could tell me what to do! ๐
*Thread Reply:* Wonderful! I'm just finishing a tutorial on how to review, and will post it to you
*Thread Reply:* ohh I thin my invitation expired. :(
*Thread Reply:* Oops I will try again
*Thread Reply:* @Barbara try again now!
*Thread Reply:* I resent it
*Thread Reply:* I will check the tutorial. I am hurry with an inform but finish that I will be more active in the analysis! I really want to lear!
*Thread Reply:* Did you fix the spelling mistakes already? You should be able to aedit the page
*Thread Reply:* No haven't, but can do next week though!
*Thread Reply:* Spent my morning converting unix commands to harry potter spells... I suppose I could do something more productive... ๐
*Thread Reply:* There was just one last thing with one of the dates, sorry... Have a look and see what you think, otherwise all done!
*Thread Reply:* @Meriam van Os I don't see any comments about dates I hadn't resolved?
*Thread Reply:* Ah it came up now. Will look this evening ๐
*Thread Reply:* Sorry, I forgot to klick on submit review...
*Thread Reply:* No worries ๐ done it myself too,
*Thread Reply:* To respond: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/439#discussion_r720768505
('ve resolved but without change)
*Thread Reply:* But actually, while that one was OK, XBQM11 so I'll update that now (I think I copy and pasted the wrong value ๐คฆ )
*Thread Reply:* And the NAs are inferred archaeologically from the rest of the text
*Thread Reply:* REquested re-review, and maybe I can do a release tomorrow if all good now ๐. Thanks for your very careful checks @Meriam van Os!
*Thread Reply:* Woohoo thank you!
@channel I'm starting to plan the metadata-thon.
For those who will be joining on Friday. November 5th, I would ask you to kindly answer some questions on the following google form for organisational purposes (timezones etc.):
if you didn't previously respond, but would since like to join and contribute: please feel free to fill in the form! More the merrier! We will be cleaning up library-level metadata and making a tool to assist in downloading the data :).
https://forms.gle/wupVap56sHuDS28q7
Aaaaand AncientMetagenomeDir release v21.09: Taputapuฤtea is now live!
Release v21.09 includes 9 new publications, representing 122 new ancient host-associated metagenome samples, 13 new ancient microbial genomes, and 12 new ancient environmental samples.
This brings the repository to a total of 132 publications, 1038 new ancient host-associated metagenome samples, 368 ancient microbial genomes, and 375 new ancient environmental samples.
And due to some GitHub magic, contributors are now listed in the given release :)
https://github.com/SPAAM-community/AncientMetagenomeDir/releases
If anyone wants to practise submitting data, here is another one: https://github.com/SPAAM-community/AncientMetagenomeDir/issues/444
*Thread Reply:* Almost as inspiring as medieval tavern ๐ป
Also slides for Next Friday: https://hackmd.io/@jfy133/HyKgEJVLY#/
If anyone wants to look through and let me know of anything unclear/confusing and I will update them before the event ๐
Update on the open data award:
We were shortlisted but didn't make it. But encouraged to apply again next year (I suspect they want to see some more real world use)
*Thread Reply:* A shortlisting is a win!! Yay for us!
*Thread Reply:* ๐๐๐I agree! Well done!
For Friday, if you're new to git this Twitter thread is a good start to get familiar with some of the different terms we will be using!
}
Twitter
Slides: https://hackmd.io/@jfy133/HyKgEJVLY#/
*Thread Reply:* Hey, looks like I can't merge it
*Thread Reply:* Which one did you resolve?
*Thread Reply:* We will have to go one by one through the 3 sequencing centres PRs
*Thread Reply:* No worries! Thanks for your contributions!!
*Thread Reply:* Thanks for coordinating this fantastic effort! I'll do a bit more tomorrow
*Thread Reply:* No worries at all !
*Thread Reply:* Only come when you have time!
*Thread Reply:* I reexcuted the tests for Schulte btw ๐
*Thread Reply:* So yu can labbel that for review hwen you have time
*Thread Reply:* Can do that while cooking ๐
@channel We should be starting in 1 minute: https://gather.town/app/biUhnBjsbwZexQ8w/ancientmetagenomedir
*Thread Reply:* I will try: your sample will be the element that you produce your extract from
*Thread Reply:* from that extract you can produce multiple libraries
*Thread Reply:* if it is not clear, come find me in gathertown ๐, I am by the raised beds
*Thread Reply:* Yeah, sample is e.g. bone or tooth, Library is the DNA from the bone/tooth with sample-specific ID tags on it
*Thread Reply:* so we put the whole โPfu Turbo Cx Hotstart DNA Polymeraseโ or I assume rather โPfu Turbo Cx Hotstartโ?
*Thread Reply:* and does anyone know the one they use for ss-libraries in Leipzig? so i save time looking it up
*Thread Reply:* โOwing to the low concentration of template DNA in these indexing PCR reactions, a hot-start enzyme, AccuPrime Pfx DNA polymerase, is used to suppress the formation of primer dimers.โ
*Thread Reply:* from the Gansauge paper
*Thread Reply:* Can you give an example?
*Thread Reply:* And what a fail if we can't even do it properly ourselves ๐
*Thread Reply:* MUR009.A0201aS0L001R1001.fastq.gz on ENA.. but there is no MUR009.A02..
*Thread Reply:* Yeah stick with what's on ENA
*Thread Reply:* can we have a quick chat.. just to be on the safe side?
*Thread Reply:* @James Fellows Yates @aidanva Could I quickly check - if multiple libraries have been used for a single run (ERR entry), and these library names are given in the supplementary information but not in the ENA database, what do we do? Do we keep 'unspecified' as in the ENA database, or do we add the library names separated by ';'?
*Thread Reply:* So all the original libraries have been merged into one FASTQ?
*Thread Reply:* From what I understand, I believe so
*Thread Reply:* Then put unspecieid
*Thread Reply:* Great, thanks!
Can someone check: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/703
https://www.sciencedirect.com/science/article/pii/S0964830521001724#bib50
@channel it's me again! You must be so happy to hear that ๐
Just a reminder than we have the Metadatathon round 2 *next week* on Thursday 16th!
Please remind yourself of the schedule and location next week: https://spaam-community.github.io/#/events/metadatathon-dec2021/README. Note the gather.town location has CHANGED.
For those who were in *round 1* please try and check your open PRs and resolve any comments the reviewers have made BEFORE the event so we can focus on new publications!
For those in Asia-Pacific time zones (mostly :flagau: :flagnz: I believe ) please let me know what time you would prefer to start (either the your Wednesday night, or your Thursday morning), as before
Otherwise:
Your GOOD MORNING from a frosty Spain! Look forward to seeing European based people in one hour!
*Thread Reply:* Liam is home with a fever today, so my participation will be limited...
*Thread Reply:* Sorry to hear that, hope he feels better soon ๐ค
https://gather.town/app/PlXjb0deog0B4JCq/spaam-community ๐
*Thread Reply:* You're still unmuted while James is talking... ๐
https://hackmd.io/@jfy133/HyKgEJVLY#/
*Thread Reply:* Hmmm good question.
*Thread Reply:* I think we should fix that after everything is ready
*Thread Reply:* Because we can automate that
Column "strand" does not exist anymore, right? Found it in none of the four TSV headers.
*Thread Reply:* No waiting. Add it and pull it over to the next column in the Project board :)
*Thread Reply:* nevermind, I read further and they also do some target capture ๐
*Thread Reply:* Ok cool!
generally you can just copy over from the ENA for that column
*Thread Reply:* But good you're checking as people don't do that properly sometimes
*Thread Reply:* Centre for GeoGenetics, University of Copenhagen, Denmark
*Thread Reply:* Drop Denmark
*Thread Reply:* but the other centers in the tsv are not as specific, so I wanted to check if I should modify it
*Thread Reply:* Section for Evolutionary Genomics, The GLOBE Institute
*Thread Reply:* Is the other one from the same group... which is probably wrong but too late I guess
*Thread Reply:* Well 'same group' isn't correct because politicis, but you get the idea
*Thread Reply:* https://github.com/SPAAM-community/AncientMetagenomeDir/blob/master/assets/enums/sequencing_center.json
*Thread Reply:* do I add Centre for GeoGenetics, University of Copenhagen to AncientMetagenomeDir/assets/enums/sequencing_center.json ?
*Thread Reply:* Yes, but with a PR made from and requested into **master
*Thread Reply:* Following on from this, should I do the same with 'University of Zurich'? I can't see it in the .json file. So make a PR from the master and requested into the master? Though interestingly, the record on EAN is 'University of Zurich', but the paper specifies 'Functional Genomics Center Zurich' (something to add to the issues encountered file?)
*Thread Reply:* Go with what is on the ENA
*Thread Reply:* Great, I'll make the PR now and go with 'University of Zurich'
*Thread Reply:* Sorry again ^^ I can't commit it to master given it's a protected branch. Would you be able to add "University of Zurich" for me?
*Thread Reply:* Which si your PR?
*Thread Reply:* ahh yes same problem for me โ๏ธ:skintone2:
*Thread Reply:* Speaking of centers, for Ahmed2018 the center name on ENA is Stockholms university (which the paper doesn't state even in the supp files, and the RAW file says NA), json file has SciLife.. What should I choose? ๐
*Thread Reply:* If RAW says NA just put Unknown (or unspecified, or unknown I can't remember... please check the README for what past-James would've said)
*Thread Reply:* Pull Request: Update sequencing_center.jsonย #753
*Thread Reply:* PR: Add Centre for GeoGenetics to seq_center.jsonย #757
*Thread Reply:* Institute of Clinical Molecular Biology (IKMB): KrauseKyora2018, Haller 2021, Susat2020
*Thread Reply:* No the enum
Martin2013 built libs with two NEB kits: NEBNext DNA Library Preparation for Illumina and NEBNext DNA Library Preparation for Roche/454. Iโm not familiar with either, but my Googling suggests the lib prep for Illumina is double stranded and for Roche/454 is single stranded. Does this seem right?
*Thread Reply:* The column name is misleading - concentration is usually a unit per volume, but this appears to want some kind of count
*Thread Reply:* What count? Total reads in library? Number of reads per some volume?
*Thread Reply:* She doesn't have a library concentration, but she has a molecules/volume column
*Thread Reply:* Maybe that's why I'm confused
*Thread Reply:* Let me check
*Thread Reply:* No it is correct I thnk
*Thread Reply:* AL concentration (copies/ ยตl)
*Thread Reply:* They basically copied what we did for Deep Evo so it sshould be the same
*Thread Reply:* I made a mistake in the title of the pull request, it's supposed to be 2020 not 2021, so I changed that too
*Thread Reply:* good question
*Thread Reply:* I would put Unknown
*Thread Reply:* Depends, but normally put as 'unknown' if they are conflicting
*Thread Reply:* Put unknown then
*Thread Reply:* Did yo mean HiSeq 4000 though?
*Thread Reply:* https://www.ebi.ac.uk/ena/browser/view/ERR1883928?show=reads
*Thread Reply:* Only pathogen
*Thread Reply:* You can just close it, or just edit the current branch to remove the host one. It will update for you - PRs are not static
@Ophรฉlie Lebrasseur don't forget to go back to your old PRs from the first event: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/677
*Thread Reply:* And also #712
*Thread Reply:* Yes! Thank you for reminding me! I've had to run away for some urgent lab work but should be done and back in 30!
*Thread Reply:* Yup I'm joining u guys again later in the day
*Thread Reply:* and sorry for not pulling my weight more - working off a crap laptop and unsure about PR
hey @thedir-metadatathon-reviewers -- the study on parovirus has no actual samples linked on ENA -- but has the algined reads on a github repo... thoughts on how to proceed? ENA: https://www.ebi.ac.uk/ena/browser/view/PRJEB26712 github: https://github.com/acorg/parvo-2018
I assume just sending a link ot a github for a given project isn't what we had in mind.. ๐
*Thread Reply:* Please skip that, I still haven't worked out a way to incorporate this in yet
*Thread Reply:* No no, not your problem
*Thread Reply:* An ealrier PR was accidently merged
*Thread Reply:* But you branched while it had gone 'unnoticed'
*Thread Reply:* Yes, it's now an error from your data ๐
*Thread Reply:* I want to, if that's okay๐
*Thread Reply:* No problem!
*Thread Reply:* I have a meeting this morning but Iโll try to come by later!
*Thread Reply:* Which one is used for the initial indexing amplification?
*Thread Reply:* Usually there are two amplification steps during library construction. The first usually uses a non-proofreading enzyme for initial indexing, and the second uses a proofreading enzyme for library completion. You want to record the initial indexing enzyme because that is the one that determines the level of damage in the downstream sequences
*Thread Reply:* They used different enzymes for diff samples it seems. "Platinum Pfx was used indexing PCR for most samples, but since this was discontinued in 2018, Platinum SuperFi was used for some samples..."
*Thread Reply:* Ahh. Do they specify which samples were processed with which? If not, then put Unknown
*Thread Reply:* They unfortunately didn't. Unknown it is.. Thank you!
*Thread Reply:* I will also be joining later so maybe Iโll catch you there Miriam:)
*Thread Reply:* ๐ I hope to see you later Che!!
*Thread Reply:* @Ian Light what exacly happened?
*Thread Reply:* i just pushed a commit to dev branch on the .tsv
*Thread Reply:* see https://github.com/SPAAM-community/AncientMetagenomeDir/commits/dev/ancientsingleg[โฆ]ated/libraries/ancientsinglegenome-hostassociated_libraries.tsv
*Thread Reply:* Oh sheeeeet
*Thread Reply:* will revert those two
*Thread Reply:* ok reverted
*Thread Reply:* You'll need to reconstruct it from the old git history
*Thread Reply:* I am getting an error message related to Wagner2014 during the pull request, do I have to do something to change that?
*Thread Reply:* Shit sorry: @Bjorn Bartholdy you'll need to reopen your PR from Rascovan, I made a booboo there trying to fix the Wagner mistake
*Thread Reply:* it also looks like some other committs in the dev branch did the same thing i did..
*Thread Reply:* (look at committs from today)
*Thread Reply:* @James Fellows Yates by reverting the merge commit?
*Thread Reply:* No don't worry: Aida will 'review' directly on dev
*Thread Reply:* Thanks very much!
*Thread Reply:* I second Kelly. Thanks a lot x2! @James Fellows Yates
*Thread Reply:* just opened a PR for the wagner data -- not sure if the automated checks ran. i will check on this later tonight or tomorrow!
Thanks for all your help and organizing this James!
Errors with ancientMetagenomeDirCheck like the one below - how do you dig into underlying issue?
@channel Once again thank you much for all your efforts yesterday! I would like to give a few stats, and also ask for some feedback to make next event(s) better/work smoother!
To give you some insights of the scale we are working, we added since the previous event:
โข Single genome host associated (pathogens): 239 -> 652 (+413) new libraries โข Metagenome environmetnal (sedaDNA): 47 -> 69 (+22) new libraries โข Metagenome host-associated (microbiome): 484 -> 545 (+61) new libraries For a grand total of 496 new libraries (note this is slightly biased by the order of the issues, with single genomes being top of the project board)
We also have started work on almost 2/3rds of the publications, which is great progress!
So ๐ ๐ to the 27 contributors who joined this time around!!
I would be very grateful if you have a spare 2 minutes (literally) to give feedback here: https://forms.gle/2GWFJax25yWm8ygo8 I will make sure to make any future events run more smoothly!
*Thread Reply:* Ah, basically dev was updated since you made your branch, and you have made changes in the same place as the update. Git doesn't know which changes to pick, which is why it says that.
It's normal :)
I'll resolve it for you in 20m or so!
Also sneak-peek on the awesome work @Maxime Borry has down for the download tool ๐ ๐
@Shreya https://github.com/SPAAM-community/AncientMetagenomeDir/pull/724 I've already reviewed this one (still unresolrved) - we did we talk previously about this paper? Is there a reason why there is only one library, but 4 strains were reported in the paper?
*Thread Reply:* we did talk about it! long enough ago that I donโt remember what we discussedโฆ will take a look later today
*Thread Reply:* sorry for missing that - but i suppose that's what the second pairs of eyes are for - when I checked the witt2021-libs branch, "Unknown" was already used as the sequencing centre - does PR #805 just need to be reviewed one final time and/or accepted?
*Thread Reply:* I updated it for you. I still need to review it properly, so you can leave it for now (I'm going from oldest to newest), but wanted to let you know why I closed the Illinois PR and why your PR would've looked different
Is there anyone on the reviewing team who could quickly approve this one?
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/825
*Thread Reply:* Mostly minor but there were so many rows I may have not been in the mood to update it myself after going through so many other PRs... sorry - nothing personal ๐
*Thread Reply:* wasnโt grimacing at you. I was grimacing at myself! Should be fixed now (after spending an hour re-learning everything I learned in December and then promptly forgot).
*Thread Reply:* No worries - was just giving clarification that it wasn't a major thing ๐
*Thread Reply:* Not a photo issue, but it looks like Raphael's name is split on 2 lines - the 'l' is on a 2nd line
*Thread Reply:* No problem with the photos, but the Irish flag is green white orange rather than the other way around ๐
*Thread Reply:* Uhh oh! That's quite a big bug in the Google emojis then ๐ฑ
*Thread Reply:* I thought it looked a bit odd but thought I was being silly
*Thread Reply:* Ooooor i accidently selected cote de ivorie ๐คฆโโ๏ธ:skintone3:. I will fix tomorrow ๐
*Thread Reply:* I think I must have searched with i and was to fast and selected the first hit
*Thread Reply:* Sorry ๐คฆโโ๏ธ:skintone3:๐คฆโโ๏ธ:skintone3:๐คฆโโ๏ธ:skintone3:
*Thread Reply:* My Photo looks a little wider than normal... Not sure. If you didn't play with the aspect ratio at all, nevermind. Hmm some others also.
*Thread Reply:* I think it's a artefact of the cropping unfortunately...
But I don't think people would study it so closely for it to be an issue. But if you don't like it let me know I'm happy to try another trick
*Thread Reply:* @Iseult that should be better ๐
*Thread Reply:* Hello ^^ Thank you for checking @James Fellows Yates! I'll send you more of a portrait than the one I currently have. Also, would you be able to add the Argentinian flag? You wouldn't have known, but I'm half based in France, half based in Argentina ๐ Ta!
*Thread Reply:* Tha'ts awesome! absolutely ๐
@Bjorn Bartholdy I changed my mind about waiting to use your PR as a guinea pig, lets get it in now ๐ did the review for you: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/634#pullrequestreview-882565535 And I'm already talking separately to @Arthur Kocher about his PR for Kocher 2021
Once they are merged in we must remember to make issues to add the libraries for those two publications too
*Thread Reply:* When will be the next release?
*Thread Reply:* Even we get the libs done, or do you want it earlier?
*Thread Reply:* I'm in no rush, not sure when I can get to it, but I also don't want to hold up the next release
*Thread Reply:* Ok, if you don't have time let me know and I can correct for you
*Thread Reply:* @James Fellows Yates I'm hoping to get to it by the end of this week, but feel free to make the changes if you also find some time!
*Thread Reply:* This week is fine
*Thread Reply:* Thanks to you @James Fellows Yates for all the fantastic and hard work!
In addition to the above: I would like some second eyes on this one - https://github.com/SPAAM-community/AncientMetagenomeDir/pull/806
I've updated it how I tihnk it should work but I had to jump across many different tables and tabs so I might have made copy paste errors (@Alex Hรผbner, @aidanva or @ร shild (Ash))maybe?
*Thread Reply:* Hello @James Fellows Yates - Just filled in the doodle - all the dates in April for me are provisional as, as you know, I don't quite know where I'll be then, but I'll do my best to make it ๐
*Thread Reply:* Also - argh I can't go back to editing my name so it's my full name ๐
*Thread Reply:* Don't worry, you i can recognise easily ๐
*Thread Reply:* I will join this time if the date that is decided on works with my schedule :)
*Thread Reply:* that would be really helpful ๐
https://app.gather.town/app/PlXjb0deog0B4JCq/spaam-community
*Thread Reply:* Sounds good, I agree.
@channel starting in 5 minutes: https://app.gather.town/app/PlXjb0deog0B4JCq/spaam-community
Slides: https://hackmd.io/@jfy133/HyKgEJVLY#
*Thread Reply:* Can oyu send a screenshot?
*Thread Reply:* Ah ok, yeah in this case trust the ENA table
*Thread Reply:* "Amplified libraries were pooled with other indexed libraries and sequenced on Illumina platforms using the Paired-End mode. Screening and library validation were carried out on a MiniSeq instrument (2x80 bp reads) at the CAGT laboratory. Libraries showing endogenous human DNA content higher than 10% were selected for deep-sequencing and sequenced on a HiSeqX (2x150 bp reads) at the CEA/CNGM (Evry, France) or on a NovaSeq S4 (2x150 bp reads) at SciLifeLab (Stockholm, Sweden). These represented 5/12 and 7/11 remains for Mont-Aimรฉ hypogea I and II respectively, 1/6 samples for Mas Rouge, 4/7 samples for Grotte des Tortues, 3/7 samples for Grotte Basse de la Vigne Perdue and 6/10 samples for Grotte du Rouquet."
*Thread Reply:* omg that's rzy ๐
*Thread Reply:* Can you send me the PRJ code?
*Thread Reply:* I want to double check something
*Thread Reply:* Ok give me 2 mmutues
*Thread Reply:* Checking now
*Thread Reply:* Go with what's on the ENA
*Thread Reply:* it corresponds to what files are available
*Thread Reply:* Unless they have uploaded both raw AND merged reads ๐ค
*Thread Reply:* The first set of data we can get to is step 5... the output from bowtie2
*Thread Reply:* Did Mรผhlemann et al. upload to GenBank or to ENA?
*Thread Reply:* but they still generated the sample upload sheet ๐คท
*Thread Reply:* So they have ERS codes, and these ARE linked to the genbank thing
*Thread Reply:* What is the question here? whether to add โUnknownโ in cases of lack of Project Accession?
*Thread Reply:* Yes, and whether it's ok to use the Genbank ID for both sample and run accession
AHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHh
*Thread Reply:* Talk aboutโฆSPAAMโฆamirite ๐
*Thread Reply:* Hi @Kelly Blevins i had similar issues with BarGal 2012.. maybe you want to have a look how i managed it there.. (with lots of help from James)
*Thread Reply:* not exactly the same, but some columns could be similar.
*Thread Reply:* Follow README as much as possible, NA is a special thing in JSON so we can only use it with Numeric columns
I have got an error during my checks which I don't quite understand:
*Thread Reply:* All good on my end, thank you ๐
*Thread Reply:* This American will probably show up long after all of you are asleep
*Thread Reply:* @Shreya if you could check if you have any remaining from previous ahckathons not finished that would be great โค๏ธ
*Thread Reply:* Just a note - I'm the only one in the space so I wouldn't stress too much ^^
*Thread Reply:* Is there anything else I can do if all the 'To Do' have been done?
*Thread Reply:* Nope! Well yes (we have some new sample level metadata to add), but I consider the main objective completed which is fantastic!
I would say take the extra free time for your own stuff instead
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/879
@irinavelsko @Alex Hรผbner @aidanva and particularly @ร shild (Ash) it would be nice to get some feedback on this: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/668/
And Ash (who I just saw is on holiday), for Mendum, I've tweeted TreeBASE asking why the website is locked... https://twitter.com/jfy133/status/1513836303946821632
Twitter
}
Twitter
*Thread Reply:* I added the reviews.
Two from me I can't review myself ๐ฌ
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/892
*Thread Reply:* If it's not clear just put Unknown
*Thread Reply:* I might drop that column as it's often a need
*Thread Reply:* Also, sometimes people send samples for commercial sequencing, which is what we do for everything
*Thread Reply:* There is no โsequencing_centerโ choice on ENA, just wondered if this the equivalent to โcenter nameโ
*Thread Reply:* > Also, sometimes people send samples for commercial sequencing, which is what we do for everything > This is also why I'm thinkibg of dropping
*Thread Reply:* Maybe replacing with codes from ancient-metagenimic-labs
*Thread Reply:* As it's the extraction/library prep which has a stronger bias than the sequencing in most cases
*Thread Reply:* I think that is a much better idea, although can get complicated when samples are generated in multiple labsโฆbut useful to know
*Thread Reply:* I might be getting a Hiwi that can help with such things ๐
Would anyone be willing to review this and see if we are missing anything: https://www.mdpi.com/1999-4915/14/6/1336/htm
Table 1
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/899
@channel we are REALLY close now! https://github.com/SPAAM-community/AncientMetagenomeDir/pull/857
Just need reviews!
*Thread Reply:* ๐คฉ You've earned such a well deserved rest!
OK clean up PRs incoming:
https://github.com/SPAAM-community/AncientMetagenomeDir/issues/732
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/903
*Thread Reply:* Took care of the first one ๐
*Thread Reply:* Friday afternoon a good time for review requests I see ๐
*Thread Reply:* Extra points for doing it during USA waking hours ๐
*Thread Reply:* @James Fellows Yates Iโve approved it, do I go ahead and merge or do you?
*Thread Reply:* You can mege
*Thread Reply:* Satisfying button ๐
*Thread Reply:* same for you @Bjorn Bartholdy
*Thread Reply:* (working out the third fix)
*Thread Reply:* I'm particualrly looking at you @Alex Hรผbner @irinavelsko @ร shild (Ash) @aidanva as often the most opinioanted ;)
*Thread Reply:* I'll make a PR into dev with the proposal in a bit if it helps
*Thread Reply:* Basically check any reference to anthropological stuff is gone other than the note in the README ๐
Ok now a tiny correction to Fotakis 2020: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/909/files
*Thread Reply:* Serves me right for manual editing!
*Thread Reply:* 2 minutes...
*Thread Reply:* But the info should now be the same as here basicaly: https://github.com/SPAAM-community/AncientMetagenomeDir/blob/7e09f7d5b69cc71afcff4649d4d0ebdbd20c1a07/ancientsinglegenome-hostassociated/libraries/ancientsinglegenome-hostassociated_libraries.tsv#L1089
*Thread Reply:* Ok try again ๐
*Thread Reply:* Gotta love those rogue spaces at the end of a line ๐คฃ
*Thread Reply:* Thanks very much @Bjorn Bartholdy!
*Thread Reply:* Back to making silly R plots
*Thread Reply:* Is the third one (libraries per year grouped by layout) showing what it's supposed to be showing? I don't see the groups.
*Thread Reply:* No it's not, good catch ๐ค
*Thread Reply:* Meant to be showing paired or single
*Thread Reply:* There also seems to be a mismatch between the standard cumulative plots and the grouped cumulative plots. They should contain the same number of publications, but the grouped plots seem to be missing some. The grouped plots also don't seem to be displaying the cumulative sum (right?)
*Thread Reply:* As for general feedback, it's definitely a nice way to see various trends over time!
*Thread Reply:* I just realised this morning that the group by summary may not include sets if something is NA... I'll go back and check
*Thread Reply:* Ah no it's the joining operation not making (obvs) categories with 0 when that particular year doesn't have any data
*Thread Reply:* ๐คฆโโ๏ธ:skintone3:
*Thread Reply:* please just assign me to some, iโll have time later today!
*Thread Reply:* Assigned you 3 pathogen ones!
*Thread Reply:* Assigned! More of a mix this time but should be pretty simple
*Thread Reply:* Now you have 12, @James Fellows Yates. ;)
@channel reminder that if we don't get reviews, that means no release, which means to no paper ๐ the more people who help out the less work everyone has to do ๐
https://github.com/SPAAM-community/AncientMetagenomeDir/pulls
*Thread Reply:* I can do some later today - what needs to be done exactly? if you just assign me a couple I'll get around to it in a few hours ๐
*Thread Reply:* Just check that my changes don't have typos/make sense :)
*Thread Reply:* Here you go:
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/915 (I think you were involved with this set of metadata at some point, basically there were a lot of missing libraries, so it adds the missing ones) https://github.com/SPAAM-community/AncientMetagenomeDir/pull/918 (ez-peasy typo fix) https://github.com/SPAAM-community/AncientMetagenomeDir/pull/930 (typo fixes)
*Thread Reply:* Thanks very miuch @ร shild (Ash) and @Iseult! 6 more lean ups to go ๐
*Thread Reply:* Thank you @ร shild (Ash)!
Not yet, need opinions on the figures here: https://spaam-community.slack.com/archives/C0183TC8B0R/p1657114641355499
If they look ok, if any are not useful, if there are others that should be made instead
}
*Thread Reply:* I think the plot about what type of sequencer was used would make more sense as a line plot of the usage per year rather than a cumulative sum plot. There are too many categories in the stacked bars to get something meaningful out of it other than the major trends. It is particularly hard to infer anything from the environmental samples because you prefer to use the same y-axis scale and there are the fewest libraries. So I cannot really take much from this plot. The other ones look good.
*Thread Reply:* @Alex Hรผbner does this look better
*Thread Reply:* (I clicked both, prefer 2๏ธโฃ)
*Thread Reply:* I'm in, but not for September, as I'll be in conference, and holidays
*Thread Reply:* Eek. End of august?
*Thread Reply:* I'm only free again from the beginning of October
*Thread Reply:* I am in guys, just let me know when. I can help with both R and Python depending on where needed
*Thread Reply:* I'll send when2meets on monday!
*Thread Reply:* ๐ฑ good luck with the move! Could you indicate with the emojis too?
*Thread Reply:* Indicate what? Lol sorry for not getting it ๐
*Thread Reply:* that you would help wiht R
*Thread Reply:* Cool, I'm down for both then!
*Thread Reply:* @James Fellows Yates sorry, I missed a conflict in my calendar... ๐ฌ I can still join after 13.00 if necessary!
*Thread Reply:* No worries!
*Thread Reply:* @Olivia will likely be joining around the nanyway
https://app.gather.town/app/PlXjb0deog0B4JCq/spaam-community
James is representative for all our video-off icons โ staring at our code/tables trying to understand why it's behaving like it is ๐
*Thread Reply:* @Olivia we have put all the ones that need fixing, at the bottom you will see the ones you are working on, you can then tick them when you have solve them ๐
*Thread Reply:* I donโt seem to have access to the file!
*Thread Reply:* you would need to log in, with you git login should work
Perhaps overlaps a bit with what the visualization team is doing, but here are some heatmaps of missingness in the three data sets (environmental, metagenome, singlegenome) ordered by publication date. The values in the parenthesis for each variable indicate percentage of missing data. Would this be useful?
*Thread Reply:* It'll vary per column - you should check the README next to each table
*Thread Reply:* Note you should pull from dev, as Aida found an inconsistency between the two tables for archive
*Thread Reply:* โข ~Moguel 2021: Additional info is shown in libraries.tsv (which isnโt found elsewhere in the metadata, so there is a usefulness to having it). Example: paper shows 200_4, samples.tsv shows 200_4 , libraries.tsv and ENA show 200_4_zone3_hyposaline~
โข Liang 2021: The samples.tsv only has 3/15 and the libraries.tsv has the other 12/15 (no overlap, the two together comprise the entirety of the ENA/paper entry).
โข ~Borry 2020: two ID types listed in the paper - archaeological ID and laboratory ID. Samples.tsv uses labID, libraries.tsv uses archID. ENA has archID as Sample Title and labID as Sample Alias~
โข Pedersen 2016: ENA sample alias and libraries.tsv file match, using style ICF40 though ENA title shows Ice-Free corridor for all. Sample.tsv shows CHL_131_11266 as sample ID, which is consistent with the paper and is also shown as the library_name in the libraries.tsv.
*Thread Reply:* โข Mougel: stick with paper/samples โข Liang: uhh that's confusing.. I will have to have a look โข Borry2020: stick with archID
*Thread Reply:* We will need to do the same thing with the host-associated metagenomes for Borry2020
*Thread Reply:* Can you do that?
*Thread Reply:* Sure, I am almost done reviewing the env ones Aida assigned me ๐ and I will do it after
*Thread Reply:* What's your github ID btw?
*Thread Reply:* ๐ gonna try something fancy (for em anyway)
*Thread Reply:* HEHEH i worked!
*Thread Reply:* opening pull request from command line requesting two reviewers ๐
*Thread Reply:* @aidanva @James Fellows Yates Iโm thinking that the CHL IDs that are in the paper make the most sense for Pedersen 2016, does that work for yโall?
*Thread Reply:* Could you join Gather?
*Thread Reply:* Aida and I are talking about it right now in fact
*Thread Reply:* @Bjorn Bartholdy https://spaam-community.slack.com/archives/C0183TC8B0R/p1661510475199579?thread_ts=1661510101.359279&cid=C0183TC8B0R
}
Just to make sure: We prefer the "Generated FASTQ files", right? The "Submitted files" are not always available and I have seen only FTP links of the "Generated FASTQ" in the tables.
*Thread Reply:* Some. There were missing libs that I added and pushed the branch to GitHub. Not enough for a draft PR. I can update you this evening (after child-pickup and dinner)
*Thread Reply:* I'll be back about 20:30 cest
*Thread Reply:* The big problem is that the number of samples is all over the place. Supp mat has 15 (37 including replicates). samples.tsv has 16, and libraries.tsv has 30. ๐ซ
@channel as we are getting ever closer to getting releases of the data and the tool, I would like to start gather affiliations for the AMDirT + library metadata paper!
Please fill in your details here: https://docs.google.com/spreadsheets/d/14uSdUlOpqA1rxY6j-k9M30rhSGxlFQtKxrerKudZRxU/edit?usp=sharing
For people from the first publication and who I've published with recently, I've copy and pasted some of your affiliation already but pleasereview it!
https://github.com/SPAAM-community/AncientMetagenomeDir/releases/tag/v22.09.2
@channel https://twitter.com/jfy133/status/1565229326467047426
Just sent on the tweet - while I could find most people, there were a few I missed. Please PM me your twitter handle if I missed you! I've also added all your twitter handles to the manuscritp affiliation list, so please also correct me if I got it wrong there
(The main people I couldn't find, if they had Twitter or I wasn't sure was: @Arthur Kocher @Meriam van Os @valentinav and @Gabriel Yaxal Ponce Soto)
Twitter
*Thread Reply:* Thank you @James Fellows Yates!
By affiliation page I mean here: https://docs.google.com/spreadsheets/d/14uSdUlOpqA1rxY6j-k9M30rhSGxlFQtKxrerKudZRxU/edit?usp=sharing
*PLEASE ADD YOUR DETAILS!*
*Thread Reply:* 2 columns "Twitter handle". Maybe you mean one of them GitHub?
*Thread Reply:* you saw nothing
As a reminder, what we are doing is very relevent! We are ahead of the game!
https://www.nature.com/articles/d41586-022-02820-7
PR for review ๐ https://github.com/SPAAM-community/AncientMetagenomeDir/pull/938/
6 Calculus Samples / 6 Libs ๐
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/985
@channel there are loads of ancient metagenomics papers coming out! We need to make sure we keep up-to-date, please please ๐ if you have a spare 30 minutes assign yourself and make a PR:
https://github.com/SPAAM-community/AncientMetagenomeDir/issues
*Thread Reply:* @Nikolay Oskolkov as you attended in the past, would you be able to join either day?
*Thread Reply:* Hi James, yes, sure, sorry for my slow response! I believe both days should work for me ๐
*Thread Reply:* AWesome thanks ๐
*Thread Reply:* @Nikolay Oskolkov @Ian Light @Yuejiao Huang @Alex Hรผbner @Olivia @Shreya @Jasmin Frangenberg @Julien Fumey could you respond with a :femaletechnologist::skintone_3: emoji that you've python experience and ๐ท if you're volunteering as a tester (pig for ginea pig, which there is no emoji shockingly...)
*Thread Reply:* Holy crap @Maxime Borry you're going to have an army
*Thread Reply:* depending on how much work is still to be done on the tools, could be good... also would be a bit more flexible for people to be there for one day or the other if necessary
*Thread Reply:* Exactly, I wouldn't expect everyone to be there all the time
*Thread Reply:* https://github.com/SPAAM-community/AMDirT/issues
*Thread Reply:* Those are everything to do
*Thread Reply:* The main missing tooling is autofilter
*Thread Reply:* Ah reason why the comment is failing is because you are coming from a fork not internatally
*Thread Reply:* you're allowed to make the branch from within the main AncientMetagenomeDir repo if you want - you're a member of the organisation already
*Thread Reply:* (to make it easier on permissions stuff)
*Thread Reply:* ah OK, forgot what the Stadard procedure was and didn't want to mess anything up ๐
*Thread Reply:* You deifnitely thought right!
*Thread Reply:* But in this case it's allowed
*Thread Reply:* just don't push to master (or delete it)
*Thread Reply:* hmm will redo the PR with a branch specific thing, tried doing a quick bugfix of running the tests locally that didnt work.
*Thread Reply:* Another thing that might warrent updating is all permutations of allowed terms in the main study focus column for samples (it throws an error for Faunal,Floral but not Floral,Faunal)
*Thread Reply:* That's tricky unforutnately
*Thread Reply:* Or rather I guess it's not ,but it will make it much more filterable if we restrict to a smaller collection (as it's more consistent)
*Thread Reply:* (I also say this as I'm already not really happy about having all of these categories, because it becomes very unstanradardiseD)
*Thread Reply:* hmm maybe a discussion also for AMDirT hackathon?
@channel reminder for everyone joining the AMDirT Hackathon we will be starting in 35 minutes: https://app.gather.town/app/PlXjb0deog0B4JCq/spaam-community
For those in Leipzig, we are in the ABI seminar room (message Maxime/I if you need picking up)
Remember that this is not just a coding hackathon, we also need testers, documenters, and will use the time to also get more of the publications in if you're not comfortable wiht coding (the normal metadata gathering you're used to to)
@channel need some thoughts on this one on behalf of @Kadir Toykan รzdoฤan.
Note anyone can give their input here, regardles if you're in the hackathon or not.
Kadir proposed adding the following paper to our environmental DNA table: https://www.sciencedirect.com/science/article/pii/S0960982221008186
Now it is shotgun data from sediment DNA. But it focuses on genomic reconstruction of human/bison/dog genomes for po(o)p-gen rather than metagenomics.
However, they did use a metagenomic tool for initial screening (Centrifuge) to identify the target species.
So question: do we consider this a metagenomic sample/paper to include in the 'Dir? Or is this too pop-genny?
*Thread Reply:* @irinavelsko @aidanva @Alex Hรผbner @ร shild (Ash)?
*Thread Reply:* @Pete Heintzman?
*Thread Reply:* Definitely a metagenomics sample, and would argue a metagenomics paper. As you say, there was a Centrifuge analysis in addition to the pop-gen stuff.
One could make a similar comment about pathogen metagenomics papers, where the primary focus is often phylogeography, locus presence/absence, selection, etc, rather than just a metagenomic community reconstruction. ๐
*Thread Reply:* But is it really a metagenomics when it's just centrifuge and that's it? They don't make any biological inference from the metagenomic data itself
But fair point about the pathogens...
*Thread Reply:* I agree with Pete that when the sample is metagenomic one, we should include it.
*Thread Reply:* But isn't all ancient DNA metagenomic?
*Thread Reply:* We would have to go back and add all ancient human shotgun genomes for example
*Thread Reply:* If we decide to include this one, then any shotgun genome from ancient samples should be added, not only from humans
*Thread Reply:* or do we consider anything coming from bone as not metagenomic?
*Thread Reply:* But this is primarily a sediment sample, not some petrous bone. So we expect a similar signal as for a sediment sample that was studied for microbes and not mammals.
*Thread Reply:* OK, then we include ๐
*Thread Reply:* As it's directly from sediment
*Thread Reply:* Perhaps a definition of a metagenomics sample could be something like: โAn ancient sample that is either expected or has been shown to contain genomic information from multiple taxaโ
*Thread Reply:* Expected would be environmental deposits (sediments, dental calculus, etc)
*Thread Reply:* Shown would be tissue from individuals where pathogens and/or microbiomes were found
*Thread Reply:* That avoids the rabbit hole of including any and all ancient samples, which may well, but not necessarily, be metagenomic.
*Thread Reply:* by that definition, shouldn't be all the teeth samples included? we normally observe an oral signal in them, even if pathogens are not found
*Thread Reply:* Good point โ do the modifications above help?
*Thread Reply:* if I understand the definition now... it has to be shown in the paper that those samples have a preserved oral signal to be included?
*Thread Reply:* Thatโs correct.
Is that consistent with how you folks decide on including a tooth sample in AncientMetagenomeDir?
*Thread Reply:* We would need to further refine the definition though, because a tooth may have a very minimal multi-taxa presence still. The oral signal is present (even if skewed), so we need to be a bit more precise I think
*Thread Reply:* And may need distinct diefinitions per table
*Thread Reply:* Because no, the proposed defintion does not fit here so far
> Is that consistent with how you folks decide on including a tooth sample in AncientMetagenomeDir?
*Thread Reply:* Any further comments: https://github.com/SPAAM-community/AncientMetagenomeDir/issues/1014#issuecomment-1308726818
@ร shild (Ash) we have a question for you for here: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/1012
*Thread Reply:* Itโs all capture data, see my reply on the pull request
*Thread Reply:* pip install -e . in the repo while in a conda environment, but have you been working off of dev @Ian Light? It should be AMDirT
*Thread Reply:* @Maxime Borry โ๏ธ
And @Alex Hรผbner @Diana for this one: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/1011
*Thread Reply:* sorry, completely forgot I did that- will get to it by the end of the week ๐
*Thread Reply:* @James Fellows Yates I can't find anything about data availability in Hider 2022- am I missing something? It says on the issue you couldn't find it either though ๐ฌ
*Thread Reply:* Let me double check
*Thread Reply:* I don't have access to the PDF, do you ?
*Thread Reply:* (that is why I couldn't find it)
*Thread Reply:* or @Jessica Hider if you're around, would you be able to share with us a PDF of the paper?
*Thread Reply:* Ok yeah I also do dont' see anything in the main text... is there anything in the SI tables?
*Thread Reply:* If you don't see anything else then you can just leave a comment on the issue saying data isn't available and close it!
*Thread Reply:* And enjoy one less thing on your TODO list ๐
*Thread Reply:* Yeah, it doesn't seem to be there... thanks for double checking! ๐
*Thread Reply:* That's sad though... But thanks for looking!!
*Thread Reply:* And responding quickly โค๏ธ
*Thread Reply:* Will be done ๐ช (as always shortly before the deadlineโฆ)
*Thread Reply:* hiya james -- my morning has been thrown a bunch of random things that need to get done, but in the afternoon (1:30) I will join !
*Thread Reply:* OK no worries, see you this afternoon
I got there in the end, just before the holiday!
https://github.com/SPAAM-community/AncientMetagenomeDir/releases/tag/v22.12.0
Release 22.12.0 is out!
And current targets for 23.03!
https://github.com/SPAAM-community/AncientMetagenomeDir/milestone/9
*Thread Reply:* Unlike Pandora, I don't think we are distinguishing between individuals but indeed explicit samples
*Thread Reply:* As e.g. in eDNA samples there is no 'individual'
*Thread Reply:* It's a not as nice indeed though when you want to get to meta analyses ๐
*Thread Reply:* but I didn't want to 'hack' the other way
Hi @James Fellows Yates, The study of Farrer2021 (https://github.com/SPAAM-community/AncientMetagenomeDir/blob/master/ancientmetagen[โฆ]ciated/libraries/ancientmetagenome-hostassociatedlibraries.tsv) is listed with the librarylayout set to PAIRED, however, but all entries have either just a single or three FastQ files (R1, R2, single). I am wondering, whether the entries that have just a single FastQ file should be set to SINGLE instead of PAIRED or was there a reasoning for setting them all to PAIRED?
*Thread Reply:* Let me have a look, IIRC this was a really wierd one
*Thread Reply:* Ah yeah, because that's what is listed on the ENA...
*Thread Reply:* https://www.ebi.ac.uk/ena/browser/view/PRJNA688065
*Thread Reply:* But agreed maybe we should fix this...
*Thread Reply:* @Sterling Wright it looks like you uploaded this, do you have any idea?
*Thread Reply:* I am not sure to be honest. I was added pretty late to the project. I will have to check with Andrew.
*Thread Reply:* I confirmed with Andrew that they are PAIRED-end. When I was instructed to upload, they only gave me those single files. Looking at it now, I am thinking some things got misplaced when Laura moved from Adelaide to Pennsylvania. I will do some investigating and see if I can find both ends of the sequences. Sorry for the confusion.
*Thread Reply:* Thank you Sterling!
*Thread Reply:* No problem. Iโm sorry for this issue.
*Thread Reply:* I confirmed with Andrew that they are paired. I am in the process of tracking down each sample. It is taking a while because the data was stored at ACAD and I am trying to parse through everything here at Penn State. It may take a bit to finish it but I will upload the forward and reverse reads for each sample.
*Thread Reply:* Awesome, thank you very much for continuing to follow up!
*Thread Reply:* Thanks @Sterling Wright!
*Thread Reply:* Moltes gracies!
*Thread Reply:* Hey @Toni de Dios Martรญnez any update?
*Thread Reply:* yes, I have the sequences processed and already preprared to be uploaded back.
Is the interactive sample map failing to render for anyone else? Trying to use it for a presentation and this is the most I can get on either Chrome or Safari!
*Thread Reply:* I've got the same in chrome
*Thread Reply:* Me neither!
*Thread Reply:* Will try to investigate
*Thread Reply:* There are still the static maps on the main readme you can use as an interior work around
*Thread Reply:* @Maxime Borry ahy ideas?
*Thread Reply:* Thank you, and no hurry! Did a quick and dirty chart off the .tsv files which is more than enough for my needs right now :partyparrot:
*Thread Reply:* Github changed their rendering URL
Review needed on: https://github.com/SPAAM-community/AncientMetagenomeDir/pull/1051 (enums for) https://github.com/SPAAM-community/AncientMetagenomeDir/pull/1052
<a href="https://github.com/SPAAM-community/AncientMetagenomeDir">SPAAM-community/AncientMetagenomeDir</a>
<a href="https://github.com/SPAAM-community/AncientMetagenomeDir">SPAAM-community/AncientMetagenomeDir</a>
*Thread Reply:* Gabriel Renaud
*Thread Reply:* Hannes Schroeder
*Thread Reply:* Michael Knapp?
*Thread Reply:* More ideas:
Verena Schuenemann Ben Vernot Martin Sikora Michael Martin Lars Fehre-nSchmitz Laura Epp
*Thread Reply:* Ok this is really hard to find women who we aren't already cllaborating with ๐
*Thread Reply:* I don't know if their publications will be relevant enough... But from our department you could maybe check out Catherine Collins (and/or Alana Alexander).
*Thread Reply:* Kathrine Eaton?
*Thread Reply:* possibly Lara Cassidy - I know she has an interest in data/metadata management and standardisation but not sure if publications relevant enough?
*Thread Reply:* That's a very good call! Been speaking with her a few months ago ๐
*Thread Reply:* Ludovic Orlando, Tom Gilbert, Felix Key?
*Thread Reply:* The first two are too high up to consider IMO, the last is @Ian Light supervisor ๐ฌ
*Thread Reply:* Thanks though! Keep the mcoming!
*Thread Reply:* Tina suggest: Mikkel Winther-Pedersen
*Thread Reply:* Maybe final list:
Mike Bunce, Katherine Eaton Lara Cassidy Mikkel Winter-Pedesen Mike Martin Hannes Schroeder
*Thread Reply:* Sorry, I havenโt had time to make my comments, but will try to do asap
*Thread Reply:* No worries! Can come during preprint period like I said ๐
Final reviewer suggestions:
*Thread Reply:* I would include the first three lines of the Results in the Methods too.
*Thread Reply:* Three sentences or lines?
*Thread Reply:* Isn't the 5000 ancient metagenomic libraries results?
*Thread Reply:* ```\textbf{Background:} Access to sample-level metadata is important when selecting public metagenomic sequencing datasets for reuse in new biological analyses. The Standards, Precautions, and Advances in Ancient Metagenomics community (SPAAM, \href{https://spaam-community.github.io}{https://spaam-community.github.io}) has previously published AncientMetagenomeDir, a collection of curated and standardised sample metadata tables for metagenomic and microbial gene datasets generated from ancient samples. However, while sample-level information is useful for identifying relevant samples for inclusion in new projects, Next Generation Sequencing (NGS) library construction and sequencing metadata are also essential for appropriately reprocessing ancient metagenomic data. Currently, recovering information for downloading and preparing such data is difficult when laboratory and bioinformatic metadata is heterogeneously recorded in prose-based publications.
\textbf{Methods:} We present an extension of AncientMetagenomeDir that provides standardised library-level metadata of ancient metagenomic samples, and the companion tool `AMDirT', which facilitates automated metadata curation and data validation, as well as rapid data filtering and downloading.
\textbf{Results:} Through a series of community-based hackathon events, AncientMetagenomeDir was extended to include standardised metadata over over 5000 ancient metagenomic libraries. The companion tool `AMDiRT' provides both GUI and CLI based access to such metadata for users from a wide range of computational backgrounds. We also report on errors with metadata reporting that appear to commonly occur during data upload and provide suggestions on how to improve the quality of data sharing by the community.
\textbf{Conclusions:} Together, both standardised metadata and tooling will help towards easier incorporation and reuse of public ancient metagenomic datasets into future analyses.```
*Thread Reply:* That's the code if you want to just change it and show me waht you mean ๐
*Thread Reply:* ok I see, yes the extension to 5000 libraries is part of the results. But I see the part of how we made it to 5000 libraries as methods (e.g. โThrough a series of community-based hackathon eventsโ). but I may be wrong.
*Thread Reply:* So move community-based hackathon events to section above :+!
71 proof changes and 22 comments ๐คฆ but it's away! lets see if they come back again
Dear James
Facilitating accessible, rapid, and appropriate processing of ancient metagenomic data with AMDirT
Borry M, Forsythe A, Andrades Valtueรฑa A, Hรผbner A, Ibrahim A, Quagliariello A, White AE, Kocher A et al.
Please click here to download the PDF proof of your F1000Research article.
Please look through the article and let me know if it requires any corrections or if you are happy for it to be published as it is. Please also confirm the following details are correct:
All author names are spelled correctly
Authors are listed in the correct order
Affiliations for all authors are accurate
The information in the Copyright section is correct
All figures and figure legends are correct
All external files, including data files are correct
All links within the article are working, and correct
Please note that connecting an ORCID account to F1000Research requires the account holder to sign in to both F1000Research and ORCID, therefore it isn't possible for us to add ORCID badges for your co-authors on their behalf. When the article is published, they will receive an email encouraging them to connect their ORCID account to F1000Research. If they do this, their ORCID badge will be displayed next to their name.
Corrections at this stage may require further typesetting and therefore cause some delays. If any corrections are necessary, please mark them directly on the PDF file using the commenting and markup tools in software such as Adobe Reader.
Please return your proof corrections to us via email - please note that after the article has been published, any requests for minor corrections will only be considered on a case-by-case basis. Therefore, we encourage you to check your proofs carefully at this stage.
If there are any outstanding queries on your reviewer suggestions, then we will be in touch with you shortly.
Best regards,
Jessica
The Editorial Team, F1000Researc
*Thread Reply:* My affiliation isn't correct anymore (it was when I looked at the first version). My name should have 16, Anatomy Department, University of Otago, but my name says 27. And @Ian Light's name has 16 now instead. ๐ค
*Thread Reply:* My affiliations are also incorrect now - they should be 38 and 39 rather than 17 and 18 ๐
*Thread Reply:* Kevin's is also incorrect and should be 38 not 17
*Thread Reply:* Also my affiliation (and Nihan's) is wrong. 21 rather than 32. Thanks James
*Thread Reply:* Hi James, found these typos: Page 3 line 3: 'AMDiRT' should be 'AMDirTโ Page 5 line 2: line should end with a point, not comma. Page 10 line 14: โgreater or equalsโ should be โgreater than or equalsโ Page 11 line 38: โheaders of theโ is redundant. Page 13 line 5: โancient DNAโ should be aDNA
*Thread Reply:* Thank you very much - ๐ฆ ๐!
I'll see if they will accept these in the next round of the proof once they've fixed the stupid affiliations
*Thread Reply:* (I made have [professionally] rage replied )
*Thread Reply:* Hi @James Fellows Yates, I read the proofs (quit carefully) and did not notice any major bugs except for the affiliations ๐ you guys did a very good job with the text! If I were to pick on small details:
- page 7, in the sentence "To install, users are recommended to use the pip or Bioconda package managers", I would make "pip" and perhaps "Bioconda" italic, or use quotes to highlight it somehow, as it is a tool / terminology and (I suppose) not a regular English word ๐
- page 9, figure 2: the images seem too small to me, I can barely see the menu items that the cursor points to, but I understand that it is hard to make the size and quality much better here.
- page 12: in the sentence "For example, mapped BAM files include only reads mapped to a particular reference genome and thus do not represent a full metagenomic dataset.", I am not sure that this is true. I guess it depends on the aligner. At least, if no samtools filtering with respect to MAPQ is done afterwards, I am pretty sure that Bowtie2 and BWA (as well as e.g. STAR, just on the top of my head) include all reads, i.e. mapped and unmapped, into their BAM-alignments. So in theory, unfiltered BAM should be equivalent to raw fastq. But in general I agree with the message that fastq should absolutely be preferred for data sharing compared to BAM ๐
*Thread Reply:* Hi Nikolay,
I'll try and incorporate the changes if we get another chance
- Ok, sounds easy enough
- The HTML version (which is the main one for F1000 as it's not a Print journal) can go full size so you will be able to see :)
- You are right, I think I even mention the uBAM format, but I didn't want to get into too much detail, mainly to avoid people thinking it's alright when the fastq should be standard and what most metagenomic tools accept
*Thread Reply:* @Nikolay Oskolkov I've just looked through the proofs again
Re 1. I realized we've been not done that for any other tools or libaries throughtout the text, so I'm reluctant to go and modify all of them now (as technically we shouldn't be making any changes other than major ones...).
I think it's not too bad as in the next 5 lines we show specific commands from the tools, so I think even if it's unfamiliar with a reader, they should be able to infer what they are.
That said, reading that section did make me notice we had a separate mistake where we called it a 'Bioconda package manager', where bioconda is the repository not the manager itself, so I will correct that with the updates from Nihan.
*Thread Reply:* Sounds good @James Fellows Yates! Also if not too late I would replace "ameta" in the references with "aMeta" as this is how we typically write it ๐
*Thread Reply:* It already it is?
*Thread Reply:* Oh in hte references
*Thread Reply:* That's odd... I pulled it via cross-ref...
*Thread Reply:* Hi James sorry, I am looking at this now. I hope everything is fixed bc I saw our affiliation is also messed up.
@channel
Ok attempt FOUR for the AMDirT manuscript!
I've been communicating with a very nice and responsive production editor now (so better experience now ;)), and apparently we've found a few bugs in their production platform that actually affected other manuscripts too ๐.
Here is the latest version, where apparently the correct affiliations , should be associated with the right person. However for those with multiple affiliations: apparently their system does not allow for the order of the affiliations within each person's to go in the order we originally requested, as it forces alphabetical order. So please note that that will be not be possible but given its clearly alphabetical I hope that is not a problem. If it is please provide a workaround by re-wording your affiliation.
Aida and I are now travelling for the next 3 days, so I won't be able to look myself until probably Monday, so please take the chance that your affilTions are correct in the meantime!!
*Thread Reply:* Alas, my name has superscript 41, which corresponds to the wrong institute.
*Thread Reply:* @James Fellows Yates my affiliations should be 22, 23 (order does not matter) and not 33, 34
*Thread Reply:* Yes, mine should be 30! Unless Iโve unknowingly been hired by the Pasteurโฆ cโest pas mal!
*Thread Reply:* Yes, I'm afraid mine are still wrong and should now be 38 and 39 ๐ฅฒ
*Thread Reply:* Yeah, mine are wrong too. They should be 2, 4 31, and 32
*Thread Reply:* Mine and Valentina's are also wrong ๐ถ Should be 21
*Thread Reply:* i hate to add to the fire, but I am with 16 which should be 37 :')
*Thread Reply:* Mine is correct! :allthethings:
*Thread Reply:* Mine is right as well ๐ just to give a good news :)
*Thread Reply:* For me the first is correct (2) and the second should be 33. The way it is now in the paper is also how is was before. I am not sure if itโs the same for others, but for my affiliations there were no corrections done, basically.
*Thread Reply:* In my case I have 13 while it should be 34
*Thread Reply:* โฆsorry James, I am now from Trento, Italy
*Thread Reply:* @James Fellows Yates nothing seems to have changed for my wrong affiliation ๐
*Thread Reply:* I have the same number 41 as before, UChicago has the same number 30 as before, but my number 41 now corresponds to a different institution from before
*Thread Reply:* The easiest would be just to manually correct the affiliations, there are not so many co-authors
*Thread Reply:* I have no control over this ;(
*Thread Reply:* I am still 33,34 while I should be 23,43
*Thread Reply:* Maybe I am doing something stupid but I get this
*Thread Reply:* I logged in with my google account credentials
*Thread Reply:* I did try with those hmmm weird
*Thread Reply:* I have the same issue as @Maria Lopopolo
*Thread Reply:* Uhh ok @Nikolay Oskolkov could you download the PDF and share it here?
*Thread Reply:* Or @Shreya if she's around!
*Thread Reply:* I'm sorry you have to go through this James.. But since the affiliations are in a different order, both of mine are wrong now.
*Thread Reply:* Do we have an original list of people and affiliations somewhere? Perhaps to send to the editors so they can at least do a preliminary check for correctness before sending us another version?
*Thread Reply:* Looks like @Gabriel Yaxal Ponce Soto @Julien Fumey and I, have also wrong affiliation (13 instead of 2 for Yaxal and I, and 13 and 20 instead of 2,9 for Julien).
*Thread Reply:* Sorry James this is a pain!
*Thread Reply:* @James Fellows Yates Even yours and mine are now messed up ๐
*Thread Reply:* Yes I'm look at it now
*Thread Reply:* and saw the same thing
*Thread Reply:* For me: 5 became 16, still 5 is written next to my name. ๐ซ Fun fun fun, sorry
*Thread Reply:* Correct! Thanks for taking care of it!
*Thread Reply:* Correct ๐ Just missing ORCID (didn't notice it before, but don't mind if it's too late for that now). https://orcid.org/0009-0004-5961-4709
*Thread Reply:* Orcids come from your f1000 account
*Thread Reply:* Not from the submission
*Thread Reply:* Make sure you use the same email on f1000 as with your orcid
*Thread Reply:* Thanks, will create one.
*Thread Reply:* Mines are correct too! Thank you !
*Thread Reply:* I confirm mines are also correct ๐
*Thread Reply:* I got a reply thsi evening sadly not:
*Thread Reply:* ```Hi James,
Thanks for the message. Unfortunately, a permanent solution hasnโt been found and the issue is still affecting live articles. We apologise for the delay, and weโll be in contact when its fixed with your full proof to check before we publish.
Many thanks,
Jess```
*Thread Reply:* Are they saying that we identified a bug that's affecting all their articles?
*Thread Reply:* WE DID INDEED ๐
*Thread Reply:* Quite fundamental actually
*Thread Reply:* ```Hi James,
Thanks for the response โ we appreciate you finding the time to check the article.
The grant info has now been corrected on our system and will pull through correctly to the final version.
The affiliations โ we are honestly just as confused by this whole thing as you are! We really do apologise for the continued errors โ I wish I had a better explanation for you other than weโre still investigating and trying our best to fix. There seems to be something happening in the download of the pdf, both via author accessible proof links and our own production editor buttons. We see the affiliations correctly assigned (albeit in a slightly different order to your corrections), but when the proof is downloaded they seem to randomly assign themselves into a different order and thatโs what you see when you download. This is a lovely new addition to the error weโre experiencing across articles where pdf affiliations arenโt matching the system and web articles, and the development team are aware and attempting to fix.
We can download the pdf from our own system in a couple of different ways, and one of those ways should yield the affiliations that match (attachment). Weโve checked these against your list of correct affiliations and the assignments should be correct, but the alphabetical ordering seems to have just decided not to work โ potentially due to some attempted fixes by our development team. Could you check the attached that the actual assignments of the authors are correct (and if not please indicate which ones are incorrect)? Weโre also interested to know how important the actual ordering of the affiliations is per author i.e. when an authors is assigned to 2 affils. Are you happy for any list order so long as they appear at least in order of appearance in the list, and assigned entirely correctly?
We definitely wonโt be putting this article live with this mis-matching issue still happening, and will ensure we have your approval of the affiliation list before we do.
Thanks for your continued understanding and patience, and apologies again,
Jess```
*Thread Reply:* And earlier message from her
*Thread Reply:* We just claim a bug bounty compensation, maybe we can ask for no APC ๐
*Thread Reply:* Likely it's a subcontractor so I doubt that will go through
https://github.com/SPAAM-community/AncientMetagenomeDir/pull/1088
<a href="https://github.com/SPAAM-community/AncientMetagenomeDir">SPAAM-community/AncientMetagenomeDir</a>
*Thread Reply:* I'll be on it after lunch!
*Thread Reply:* But if anyone else is willing to tke over and get it done today they will be my favourite person of teh day
@channel please check for accuracy of the numbers with your affilation - note that the order per person may not be excact (e.g. FSU comes before EVA for Maxime), but each number affilation is with the right person now I think?!
*Thread Reply:* At this point, it's also fine with me this way ๐
*Thread Reply:* Yes agreed ๐คฆโโ๏ธ:skintone3:
*Thread Reply:* Affiliations 13 and 20 are correct ๐
*Thread Reply:* Mine is good too ๐
*Thread Reply:* All good, just I my orcid is not shown (it is linked to f1000)
*Thread Reply:* If you linked it after we submitte dit won't come up until later
*Thread Reply:* @Kadir Toykan รzdoฤan @Maria Lopopolo maybe schedule tweet/toot for next week?
*Thread Reply:* (preprint though, it's not peer-reviewed!)
*Thread Reply:* Press releasing articles: Please avoid promoting articles in the media until the article has passed the open peer review process.
*Thread Reply:* Read the next sentence
*Thread Reply:* So we are published (social media allowed), not peer-reviewed (no newspapers)
*Thread Reply:* @Kadir Toykan รzdoฤan @Maria Lopopolo probably time to post!
*Thread Reply:* Ok I will post it!
*Thread Reply:* I got the environmental one
Would this be valid?
https://link.springer.com/article/10.1007/s13313-023-00936-6
*Thread Reply:* Looks reasonable: shotgun libraries with SRA raw data available.
*Thread Reply:* has someone already suggested anders bergstrom? think he did a preprint a while ago about data availability & aDNA
*Thread Reply:* Technically I was acknowledged on that preprint ๐ฌ... butttt not a coauthor... I guess we could try? I don't know who he may have published with recently
*Thread Reply:* Ah @Ophรฉlie Lebrasseur has published with him ๐
*Thread Reply:* And @Pete Heintzman
*Thread Reply:* Unless we can make some argument about it
*Thread Reply:* (not metagenomics, and on large mega papers...)
*Thread Reply:* Xuexue Liu published software for visualising ancient sample metadata if that qualifies
*Thread Reply:* Oooh yes, link?
*Thread Reply:* https://academic.oup.com/bioinformatics/article/38/16/3992/6623404 for the paper
*Thread Reply:* Ooooh yes perfect!
*Thread Reply:* You could maybe try Logan Kistler for broad ancient genomics / appreciation of large datasets, he might show mercy
*Thread Reply:* Fair I don't think anyone in the AMDirt paper worked on plants much recently...
*Thread Reply:* will do my best, but realised after the fact that there wasn't any new data in the issue I assigned myself (from this: https://www.nature.com/articles/s41467-023-41174-0#Abs1 ) - it looks like some would have previously been uploaded as part of other studies, but others are from human aDNA studies, so might not already be there. should this be uploaded at all? If so, should I just do the libraries that haven't been put into the dir, or is there a way to add data for previously published samples with a different species?
*Thread Reply:* I left a comment on the issue to Diana actually - they are new genomes so we should still list them! We just point to the original accession codes
*Thread Reply:* A previous example is Lugli https://github.com/SPAAM-community/AncientMetagenomeDir/issues/1109#issuecomment-1725871842
<a href="https://github.com/SPAAM-community/AncientMetagenomeDir">SPAAM-community/AncientMetagenomeDir</a>
*Thread Reply:* So basically add as normal: add the new genomes to the single genome table and the corresponding libraries just re-using the ERS/SRS codes/lib metadata from earlier studies if we already in clude them ๐
*Thread Reply:* For human aDNA studies, then it'll require new ERS/SRS codes and gathering those lib. metadata info
*Thread Reply:* Please let me know if that doesn't make sense, I'm running on 4h sleep atm,
*Thread Reply:* ok Iโll do my best just been quite busy over the last few days ๐
*Thread Reply:* @Cameron Ferguson I can take over yours, if that helps, but I donโt see which ones youโve signed yourself up for ๐คทโโ๏ธ:skintone4:
*Thread Reply:* @Iseult - Same applies to the Hodgkins 2023 issue - I can take over, if you like. ๐
*Thread Reply:* That would be super helpful if you've time @Diana - I can do the review if that would speed things up as well? ๐ Thanks!
*Thread Reply:* @James Fellows Yates Ok that should be all info added now, last remaining item is updating the change log. what information should I included for that?
*Thread Reply:* Project name, DOI and your GitHub tag. Look at the othersfor examples
*Thread Reply:* would I put it under added or changed seen as its updating an existing entry with missing samples?
*Thread Reply:* Ah good point, no in this case just add your name to after Dianaes
*Thread Reply:* As it's inclusion is new to this release
*Thread Reply:* should be good to go
If anyone could make an issue (and/or even better a pull request) for the following that would be awesome (dealing with a baby with a cold and 2 days with only 4h sleep ๐ญ and now the GitHub app keeps breaking my phone [!?!!?!?])
https://microbiomejournal.biomedcentral.com/articles/10.1186/s40168-023-01647-2
*Thread Reply:* Thanks @Alex Hรผbner!
(Museum) Frog gut microbiome?!s
https://github.com/SPAAM-community/AncientMetagenomeDir/issues/1116
<a href="https://github.com/SPAAM-community/AncientMetagenomeDir">SPAAM-community/AncientMetagenomeDir</a>
*Thread Reply:* As in accepted by F1000, or as in approved by the reviewer ?
*Thread Reply:* Approved by the first set of reviewers :)
*Thread Reply:* We have at least review incoming
*Thread Reply:* ok. I guess we should wait for the other review(s) to start answering then ๐
*Thread Reply:* But most of the comments are straight forward.
The only two more involved things (outside of rephrasing) is adding some form of library of metadata filtering and maybe a little community event to test on as many machines as possible installation of the tool
*Thread Reply:* The former I have at least a UI PR open
Second AMDirT review!!! Not approved right away, but comments mostly positive and more about reworking the text to satisfy his curiosity (as far as i can see). Seems to have a bug too.
It seems we have one more incoming, then we can plan how to address everything
https://f1000research.com/articles/12-926/v1#referee-response-210768
*Thread Reply:* Is there a possible date for that seminar?
*Thread Reply:* In December
*Thread Reply:* Is what I was told
*Thread Reply:* Ok will message you with more details
*Thread Reply:* Yes, we have two reviews (they are public, you can see them on the article page), but I don't have enough time to work on it at the moment
*Thread Reply:* That's no problem at all! I just wanted to check the paper status for my MSCA report. Thank you!
*Thread Reply:* Can consider it 'under revision'
*Thread Reply:* One review approved, the other tentatively approved with (minor) comments really
*Thread Reply:* If you want a more 'traditional' category
*Thread Reply:* Hi Alex, I got the Perez et al. paper!
Can someone add the following to the issues?
https://www.nature.com/articles/s41564-023-01527-3
*Thread Reply:* And @Abby Gancz @Sterling Wright can you already guess the two things I'm going to ask ๐ ๐ฌ
*Thread Reply:* I'll create issue in a moment.
*Thread Reply:* Hey @James Fellows Yates yep I'll get to it as soon as I can. Next week okay?
*Thread Reply:* Also uploading the unmerged/untrimmed reads?
*Thread Reply:* That might take a little longer but it sounds like Laura is good with us doing that
*Thread Reply:* That would be wonderful thank you to all :) really helps with FAIRness of the dats ๐ช
Hi @channel
We've noticed a marked drop off in the amount of contributors/submissions over the year, and I would like to better understand why.
I've made a very short(! - max 2-3 minutes of your time) anonymous survey where you could give feedback. There is one multiple choice question, and one free text question. Neither are mandatory so you can pick either or.
Any and as much feedback as possible will be gratefully received to help make your lives easier and maintain the project over a longer period.
You can get to the anonymous form here: https://form.jotform.com/233351873343052
Another to add: https://www.nature.com/articles/s41598-023-48762-6
https://www.nature.com/articles/s41586-023-06965-x
*Thread Reply:* i can grab this one sometime this week ๐
Palaeofaeces if anyone is interested
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0295924
As James mentioned a while ago, there is a drop in the participation of the project (https://spaam-community.slack.com/archives/C0183TC8B0R/p1701505117017479). Some of you filled out the form and gave us feedback why this is the case. Based on your feedback, James and I have come to the conclusion that we should have a open discussion about the next steps of AncientMetagenomeDir. We will send out another message to schedule such a meeting in the next weeks.
https://www.nature.com/articles/s41598-024-52422-8
*Thread Reply:* @Anan Ibrahim @Kadir Toykan รzdoฤan you both are assigned to one each, do you think you can still make it by end of the week?
*Thread Reply:* yup, will be done by tom. ๐
*Thread Reply:* Yes, sorry, I made a mistake but Iโll delete one.
*Thread Reply:* Ah ok, then I wait ๐
*Thread Reply:* It should be correct now! But please let me know if anything needs changing ๐
*Thread Reply:* Ok will do! You're missing a CHANGELOG file update
*Thread Reply:* But otherwise:
- I' m going to try something quickly, please ignore the next two comments
- I will aim to review tomorrow :()
*Thread Reply:* Thanks! I managed to make a bit of a mess of it but Iโve added the changelog update and it should be good now ๐
*Thread Reply:* Looks ready for review ๐
*Thread Reply:* @Anna White how did you get the exact lat:lon?
*Thread Reply:* From their first publication on these samples, they have the exact location in the supplementary files.
*Thread Reply:* The pop-gen one?
*Thread Reply:* Such a weird way of doing it with those maps...
*Thread Reply:* But ok thank yo ๐ it otherwise looked correct, was just surprised how specific it was ๐ฌ
*Thread Reply:* (that's unusaual)
*Thread Reply:* Haha, yes, it was a bit of a search but that was the only mention of a specific location I could find.
@channel hope you had a good long weekend if you/your country partakes!
Reminder we have a release scheduled *end of this week*, so please open/finish your PRs, I will do a reviewing blitz on Friday and do the release then (maybe with @Alex Hรผbner if he's free/around). We still have 3 issues/publications not yet assigned (two pathogens, one metagenome)!
Also please see the post from Alex on #general https://spaam-community.slack.com/archives/CPYE64ZC6/p1711540972353019 and fill in the when2meet attendance will be important for discussing the future of the project!
}
https://spaam-community.slack.com/archives/C03BACY0CHF/p1714987640567319
}
*Thread Reply:* I sincerely apologise to everyone for missing the meeting just now.
*Thread Reply:* Sorry from my part as well! I was away on holiday 8th and 9th (which were tentative plans that I forgot to add to my calendar, so I'm also sorry if I indicated that I was available!)
Typeset revision of AMDirT/AncientMetagenomeDir2 paper!
*Thread Reply:* Here are my comments @James Fellows Yates
*Thread Reply:* Thank you @Maxime Borry! Will add to the list, I also found the wierd broken link thing
*Thread Reply:* I don't htink I'll ask them to put the 'provided here' link though...
*Thread Reply:* have you seen it? ๐
*Thread Reply:* https://github.com/10XGenomics/supernova/blob/b16b613b14687efe20a6ad33161cb461dae04699/tenkit/lib/python/tenkit/illumina_instrument.py#L12-L45
<a href="https://github.com/10XGenomics/supernova">10XGenomics/supernova</a>
*Thread Reply:* Changes sent!