問題及應(yīng)對措施結(jié)語提綱digitalcuration的興起digital-國家圖書館

1、Digital Curation for the Big Data Sciences大數(shù)據(jù)科研中的數(shù)字保存,張智雄中國科學(xué)院國家科學(xué)圖書館,提綱,Digital Curation的興起Digital Curation是什么？Digital Curation和Preservation不同？大數(shù)據(jù)科研帶來的Digital Curation挑戰(zhàn)、問題及應(yīng)對措施結(jié)語,提綱,Digital Curation的興起Digital C

2、uration是什么？Digital Curation和Preservation不同？大數(shù)據(jù)科研帶來的Digital Curation挑戰(zhàn)、問題及應(yīng)對措施結(jié)語,1、Digital Curation的興起,Data Deluge,1、Digital Curation的興起,From Data Deluge to Data Curation Philip Lord, Alison Macdonald, Liz Lyon, Davi

3、d Giaretta The Digital Archiving Consultancy Limited and the Digital Curation Centre,1、Digital Curation的興起,The Digital Curation Centre成立在e-Science Core項目的支持下，DCC于2004年3月1日成立總部位于Edinburgh的National e-Science CentreUniv

4、ersity of Edinburgh (lead,Informatics, Law, Information Services and research institutes) University of Glasgow (HATII and Information Services) UKOLN, University of Bath Council for the Central Laboratory of the Rese

5、arch Councils (CCLRC),1、Digital Curation的興起,會議期刊International Digital Curation Conference，Bath，Sep. 29 - 30, 2005 8th International Digital Curation Conference, Amsterdam, 14 - 17 January 2013 DigCCurr 2007、DigCCurr 2

6、009、DigCCurr 2013An International Symposium on Digital Curation（April 18-20, 2007）Digital Curation Practice, Promise and Prospects（April 1-3, 2009）Chapel Hill, North Carolina, United States Public Symposium, 2010-201

7、3International Journal of Digital Curation2006開始http://www.ijdc.net/Vol 8, No 1 (2013),1、Digital Curation的興起,以Curation命名的機構(gòu)The Greek Digital Curation Unit (DCU) at the Athena Research Centre(2007)UC3，University of

8、California Curation Center (2010)The Digital Research and Curation Center at The Johns Hopkins University’s Sheridan LibrariesThe University of Toronto’s iSchool established The Digital Curation Institute （ 2010 ）Purd

9、ue University Library’s Distributed Data Curation Center (D2C2) （2009）......,1、Digital Curation的興起,與Curation相關(guān)的教育培訓(xùn)DigCCurr I (2006-09)，DigCCurr II (2008-13)School of Information and Library Science (SILS) University

10、of North Carolina at Chapel Hill,NARA Preserving Access to Our Digital Future: Building an International Digital Curation Curriculum. Extending an International Digital Curation Curriculum to Doctoral Students and Prac

11、titioners International Data curation Education Action (IDEA) Working GroupDeveloping an International Curation and Preservation Training and Education RoadmapEducation for Digital Stewardship: Librarians, Archivists

12、or Curators?" Masters Programme in Digital Curation, Luleå University of TechnologyIFLA, 2011“ Education for Digital Curation” Board on Research Data and InformationSymposium on Digital Curation in the Era

13、of Big Data:Career Opportunities and Educational Requirements,1、Digital Curation的興起,相關(guān)技術(shù)工具Data Asset Framework (DAF)enumerating and auditing data holdingsDRAMBORAself-assessment of possible riskTRACTrustworthy Repo

14、sitories Audit & Certification, Criteria and ChecklistDigital Preservation Suitepreservation plansDROIDidentifies file formats......,提綱,Digital Curation的興起Digital Curation是什么？Digital Curation和Preservation不同？大

15、數(shù)據(jù)科研帶來的Digital Curation挑戰(zhàn)、問題及應(yīng)用措施結(jié)論,2、Digital Curation是什么,先說一下數(shù)字保存（Digital Preservation）數(shù)字是一把的雙刃劍優(yōu)點方便易用、可復(fù)制、易傳輸、大量攜帶...問題脆弱性刪除、盜取、修改、失真....依賴性技術(shù)、系統(tǒng)、標準、軟件、上下文（元數(shù)據(jù)）、組織、經(jīng)濟...飛速退化性（obsolescence）媒體、硬件、軟件、格式...,2、Di

16、gital Curation是什么,Digital Preservation1996年5月1日,成為重要關(guān)注內(nèi)容Preserving Digital Information: Report of the Task Force on Archiving of Digital InformationCommission on Preservation and AccessResearch Libraries Group. Inc.

17、(RLG)目標:“continued access indefinitely into the future of records stored in digital electronic form.”,http://www.clir.org/pubs/reports/pub63/reports/pub63watersgarrett.pdf,2、Digital Curation是什么,21世紀初數(shù)字保存(DP)已經(jīng)成為數(shù)字圖書館的

18、一個重要領(lǐng)域主要研究內(nèi)容保存策略和方法、保存元數(shù)據(jù)、存儲體系、保存?zhèn)}儲、保存工作流、Web存檔、保存信息模型主要標準規(guī)范：開放檔案信息系統(tǒng)（OAIS2002)、主要數(shù)字保存系統(tǒng)和服務(wù)體系e-Depot DIAS, NDIIPP, LOCKSS, Portico, CDL DPR，F(xiàn)CLA DAITSS......,2、Digital Curation是什么,為什么還會出現(xiàn)Digital Curation?已經(jīng)數(shù)字保存已經(jīng)有

19、兩個接受的術(shù)語了數(shù)字保存（Digital Preservation）數(shù)字存檔（Digital Archiving）為什么還要提出Digital Curation？Digital Curation是什么？與Digital Preservation 有什么不同的思路和方法？,2、Digital Curation是什么,Digital Curation：被創(chuàng)造的新詞Digital Data Curation Task ForceR

20、eport of the Task Force Strategy Discussion DayTuesday, 26th,November 2002,Centre Point, London WC1，January 2003 e-Science Curation ReportData curation for e-Science in the UK: an audit to establish requirements for f

21、uture curation and provision，2003 JCSR(the Joint Information Systems Committee’s Committee for the Support of Research，JISC研究支持委員會),2、Digital Curation是什么,Digital Data Curation Task Force由Tony Hey，當時JCSR的主席召集 目標：明確和構(gòu)建

22、英國原始研究數(shù)據(jù)的Curation戰(zhàn)略會議日期 2002年11月26日The application of the term “curation” is new, and in several ways the meeting found itself grappling with questions of scope, with frequent overlap with questions relating to digital

23、 preservation.It did not reach a definition of the term.,2、Digital Curation是什么,Digital Data Curation Task ForceWhat is curation? Dr John Taylor, Director General of the Research CouncilsTony Hey, distinguish the acti

24、ons involved in caring for digital data beyond its original use, from digital preservation.Seamus Ross, “curation in the museum sense” covers three core concepts: conservation, preservation and accessAlison Allden “cur

25、ation” implied in an active management of information, involving planning. re-use of data is a core issue. If data is to be re-used, then it needs special treatmentRolf Apweiler, curation is when people add value to da

26、taJeremy Frey, curation is research work in itself - managing, improving, enhancing data,2、Digital Curation是什么,e-Science Curation Report“curation” 來源于 “curator”somebody who keeps something for the public good, whose v

27、alue often needs to be brought out by the curator. 兩個重要特點more support for explicit policies with regard to data sharingdigital curator is store-keeper， but he should take an active role in promoting and adding value t

28、o his holdings,2、Digital Curation是什么,e-Science Curation Report此前“curation” is commonly used to refer to the work done on genomic and proteomic databases, annotating and managing annotations現(xiàn)在It covers a wider context

29、 than just archiving; it embraces the care of the record within scientific context and environment,2、Digital Curation是什么,e-Science Curation ReportWorking definitionsCuration: The activity of, managing and promoting t

30、he use of data from its point of creation, to ensure it is fit for contemporary purpose, and available for discovery and re-use. For dynamic datasets this may mean continuous enrichment or updating to keep it fit for pu

31、rpose. Higher levels of curation will also involve maintaining links with annotation and with other published materialsArchiving: A curation activity which ensures that data is properly selected, stored, can be accesse

32、d and that its logical and physical integrity is maintained over time, including security and authenticityPreservation: An activity within archiving in which specific items of data are maintained over time so that they

33、 can still be accessed and understood through changes in technology,2、Digital Curation是什么,e-Science Curation ReportThat the objective of digital curation of primary research data isto keep data which is valuable, poten

34、tially valuable or which is required to be kept; and in such a way that it is accessible and usable by others (while observing relevant restrictions), that its value is maintained and, where possible, enhanced; and th

35、at this activity and service should be provided at affordable and justifiable cost.,2、Digital Curation是什么,JISC通訊定義JISC circular 6/03 (Revised), July 2003The term ‘digital curation’ is increasingly being used for the ac

36、tions needed to maintain and utilise digital data and research results over their entire life-cycle for current and future generations of users.,2、Digital Curation是什么,DDC定義1DCC Approach to Digital Curation, 15 Aug 2004

37、curation : general term - taking care of things data curation : looking after and adding value to data digital curation : looking after and somehow "adding value" to digital data. This probably implies crea

38、ting some new data from the existing, in order to make the latter more useful and "fit for purpose".,2、Digital Curation是什么,DDC定義2 DCC Charter and Statement of PrinciplesWhat is digital curation?Digital curat

39、ion is maintaining and adding value to a trusted body of digital research data for current and future use; it encompasses the active management of data throughout the research lifecycle.,http://www.dcc.ac.uk/about-us/dcc

40、-charter/dcc-charter-and-statement-principles,2、Digital Curation是什么,DDC定義3Digital curation involves maintaining, preserving and adding value to digital research data throughout its lifecycle.The active management of re

41、search data reduces threats to their long-term research value and mitigates the risk of digital obsolescence. Meanwhile, curated data in trusted digital repositories may be shared among the wider UK research community.A

42、s well as reducing duplication of effort in research data creation, curation enhances the long-term value of existing data by making it available for further high quality research,http://www.dcc.ac.uk/digital-curation/wh

43、at-digital-curation,2、Digital Curation是什么,DDC定義4DCC Briefing PapersDigital curation is the management and preservation of digital data over the long-term.All activities involved in managing data from planning its crea

44、tion, best practice in digitisation and documentation, and ensuring its availability and suitability for discovery and re-use in the future are part of digital curation.Digital curation can also include managing vast da

45、ta sets for daily use, for example ensuring that they can be searched and continue to be readable.Digital curation is therefore applicable to a large range of professional situations from the beginning of the informatio

46、n life-cycle to the end; digitisers, metadata creators, funders, policy-makers, and repository managers to name a few examples,http://www.dcc.ac.uk/resources/briefing-papers/introduction-curation,提綱,Digital Curation的興起D

47、igital Curation是什么？Digital Curation和Preservation不同？大數(shù)據(jù)科研帶來的Digital Curation挑戰(zhàn)、問題及應(yīng)用措施結(jié)論,3、Curation和Preservation不同？,JISC Preservation和Curation對比JISC Digital Preservation briefing paperDigital preservationactions and

48、 interventions ensure continued and reliable access to authentic digital objects for as long as they are deemed to be of value. Digital curationmaintaining and adding value to a trusted body of digital information for

49、 future and current use; active management and appraisal of data over the entire life cycle. builds upon the underlying concepts of digital preservationemphasising opportunities for added value and knowledge through a

50、nnotation and continuing resource management.,http://sitecore.jisc.ac.uk/publications/briefingpapers/2006/pub_digipreservationbp.aspx,3、Curation和Preservation不同？,ARL的兩者對比New Roles for New Times: Digital Curation for Pres

51、ervation, March 2011Digital curation refers to the actions people take to maintain and add value to digital information over its lifecycle, including the processes used when creating digital content.Digital preservatio

52、n focuses on the “series of managed activities necessary to ensure continued access to digital materials for as long as necessary.” intersection of these actions, digital curation facilitate the preservation.,3、Curation

53、和Preservation不同？,Digital Curation: The Emergence of a New Discipline中的對比digital preservation efforts originally focussed on ensuring that material survived technical obsolescence and organisational mismanagement. Preser

54、vation implied a passive state, where material would be mothballed in an inaccessible “dark archive”, with only a few authorised users, to ensure that it retained its integrity and authenticityensuring that digital mate

55、rial is managed throughout its lifecycle so that it remains accessible to those who need to use it. Metadata is used to both improve accessibility and discoverability; and to control authentication procedures, creating a

56、udit trails to ensure that material cannot be accessed or altered by those not authorised to do so. Digital material is actively preserved, used and reused for new purposes, creating new materials. This is Digital Curati

57、on: the management and preservation of digital material to ensure accessibility over the long-term,3、Curation和Preservation不同？,應(yīng)對的問題不同Preservation應(yīng)對技術(shù)退化和組織失效CurationFrom Data Deluge to Data Curation, Data volumes, co

58、mplexity of the data itself,3、Curation和Preservation不同？,行動的目的不同Preservation以數(shù)據(jù)的生存為目的保證數(shù)據(jù)完整性、可信賴、真實性Curation以數(shù)據(jù)能夠被科研利用為目的實現(xiàn)數(shù)據(jù)管理并使數(shù)據(jù)增值,3、Curation和Preservation不同？,達成的目標Preservation使數(shù)據(jù)可訪問、可理解、可應(yīng)用Curation對數(shù)據(jù)的整個生命周期

59、進行管理，包括數(shù)據(jù)的創(chuàng)建和在舊數(shù)據(jù)之上新生成的新數(shù)據(jù)，實現(xiàn)數(shù)據(jù)利用和再生,3、Curation和Preservation不同？,為什么人服務(wù)？Preservation為了未來后世能夠利用Curation為了當前和未來可用,3、Curation和Preservation不同？,行為模型PreservationOAIS參考模型CurationDCC Curation Lifecycle Model,3、Curation和

60、Preservation不同？,OAIS參考模型6項功能活動、3類信息包、3種角色,3、Curation和Preservation不同？,DCC Curation Lifecycle ModelFull Lifecycle ActionsDescription and Representation InformationPreservation PlanningCommunity Watch and Participation

61、Curate and PreserveSequential ActionsConceptualiseCreate or ReceiveAppraise and SelectIngestPreservation ActionStoreAccess, Use and ReuseTransformOccasional ActionsDisposeReappraiseMigrate,3、Curation和Preser

62、vation不同？,活動參與成員Preservation數(shù)據(jù)提供者、數(shù)據(jù)保存者、受權(quán)使用者Curation數(shù)據(jù)創(chuàng)造者、數(shù)據(jù)提供者、數(shù)據(jù)存檔者、數(shù)據(jù)消費者,3、Curation和Preservation不同？,保存的周期Preservation從數(shù)據(jù)提供開始，一直到所要求的未來時段，保證數(shù)據(jù)生存Curation從數(shù)據(jù)的產(chǎn)生開始，數(shù)據(jù)整個生命周期，中間有丟棄,1、從數(shù)字保存到數(shù)字保管,數(shù)據(jù)應(yīng)用范圍Preservatio

63、n受權(quán)訪問Curation數(shù)據(jù)共享、數(shù)據(jù)重用,3、Curation和Preservation不同？,思路方法Preservation遷移、仿真 Curationcreation and managementadd value to generate new sources of information and knowledg,3、Curation和Preservation不同？,保存中的主觀能動性Preser

64、vationPreservation implied a passive stateCurationDigital material is actively preservedactive management of data throughout the research lifecycle.active management and appraisal of data over the entire life cycle

65、.,3、Curation和Preservation不同？,保存的地方Preservationinaccessible “dark archive”CurationOpen Trusted Repositories,提綱,Digital Curation的興起Digital Curation是什么？Digital Curation和Preservation不同？大數(shù)據(jù)科研帶來的Digital Curation挑戰(zhàn)、問題及應(yīng)

66、對措施結(jié)語,4、Digital Curation挑戰(zhàn),e-Science Curation Report,4、Digital Curation挑戰(zhàn),e-Science Curation Report,4、Digital Curation挑戰(zhàn),e-Science Curation Report,4、Digital Curation挑戰(zhàn),Data Tsunami、Data deluge、超規(guī)模數(shù)據(jù)CERN（歐洲核能研究組織）ESA（歐

67、洲航天局）未來數(shù)據(jù)規(guī)模將更大，數(shù)據(jù)增長將更快天文觀測數(shù)據(jù)Sloan Digital Sky Survey，2008年的前10年，產(chǎn)生25 terabytes數(shù)據(jù)2014，Large Synoptic Survey Telescope每晚20 terabytes2019年，Square Kilometre Array radio telescope將產(chǎn)生50 TB已處理的數(shù)據(jù)，如果以裸數(shù)據(jù)為計，每秒7000TB,4、Digita

68、l Curation挑戰(zhàn),Big Data——>big data science“大數(shù)據(jù)科研”的時代已經(jīng)來臨不僅限于大裝置或部分領(lǐng)域的科學(xué)大數(shù)據(jù)科研是一種新的科學(xué)發(fā)現(xiàn)范式Data-intensive Science，Data-intensive Discovery存在于所有科研領(lǐng)域觀測、試驗和計算機產(chǎn)生數(shù)據(jù)日益增長的價值不論是物理科學(xué)、人文科學(xué)，還是社科科學(xué)。,4、Digital Curation挑戰(zhàn),Data as

69、 the Infrastructure European Union“In a sense, the physical and technical infrastructure becomes invisible and the data themselves become the infrastructure a valuable asset, on which science, technology, the economy a